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Abstract 

The  objective  of  this  study  was  to  determine  if  the 
ability  of  Air  Force  organizations  to  contract  with  Small 
and  Disadvantaged  Businesses  (SOBs)  depended  on  the  type  of 
contracting  effort  or  the  type  of  industry  or  product  being 
contracted  for.  The  type  of  contracting  effort  was  broken 
down  into  four  major  categories  based  on  the  Federal  Supply 
Classification  (FSC)  code.  The  type  of  industry  or  product 
was  broken  down  into  five  major  classifications  based  on  the 
Supply  and  Equipment  Classification  (SEC)  code.  The  data  to 
support  the  analysis  was  gathered  from  the  Air  Force  Systems 
Command  (AFSC)  for  the  years  1987  through  1989.  The 
objective  was  accomplished  through  a  statistical  analysis 
which  included  a  confidence  interval  analysis  and  a  pairwise 
comparison  of  means.  The  results  showed  that  the 
organization's  ability  to  contract  with  SDBs  did  depend  on 
the  two  factors.  The  weakest  area  was  electronic 
contracting  within  the  supply  and  equipment  classification. 
The  strongest  area  was  service  and  construction  contracting 
within  the  aircraft  and  aircraft  engine  classification. 
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AN  ANALYSIS  OF  THE 


SMALL  AND  DISADVANTAGED  BUSINESS  SET-ASIDE  PROGRAM 

IN  THE  US  AIR  FORCE 

I,  Introduction 


General  Issue 

Throughout  its  history,  the  US  government  has  passed 
legislation  designed  to  aid  the  people  of  our  country  both 
socially  and  economically.  The  programs  created  by  this 
legislation  are  called  socio-economic  programs.  Some 
examples  of  these  programs  are  Medicaid/Medicare, 
unemployment  benefits,  social  security,  and  farm  subsidies. 
Another  program  not  so  well-known  is  the  government's  Small 
and  Disadvantaged  Business  (SDB)  set-aside  program. 

Through  this  program,  federal  agencies  such  as  the 
Department  of  Defense  (DOD),  Department  of  Energy, 

Department  of  Transportation,  and  the  Department  of 
Agriculture  are  now  required  to  set  aside  a  specified  amount 
of  their  business  for  firms  that  qualify  as  SDBs . 

The  roots  of  this  program  go  back  to  socio-economic 
legislation  known  as  the  Small  Business  Program.  The  Small 
Business  Program  was  originally  intended  to  allow  federal 
agencies  special  purchasing  provisions  in  order  to  aid 
businesses,  which  had  a  small  chance  of  becoming  successful 
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enterprises  because  of  discriminatory  practices,  to  compete 
in  the  US  market.  Since  then.  Congress  has  passed 
additional  legislation  at  various  times  to  increase  the 
amount  of  business  going  to  specific  small  businesses. 
Lately,  some  of  this  legislation  has  become  a  source  of 
controversy  because  of  additional  requirements  which  were 
passed  in  1986  and  1987  which  specifically  mandated  the  DOD 
attempt  to  set  aside  a  specific  amount  (five  percent)  of  its 
business  for  small,  disadvantaged  businesses. 

Specific  Problem 

The  DOD  passed  this  goal  to  its  respective  services.  As 
a  result,  the  US  Air  Force  (AF)  is  requiring  all  of  its 
organizations  to  meet  the  goal,  regardless  of  the  types  of 
contracting  they  perform,  the  industries  they  contract  with, 
or  their  ability  to  meet  the  goal.  In  1978,  the  goals  were 
established  by  the  head  or  each  agency  (the  DOD  is  an 
agency,  as  is  the  Department  of  Transportation,  Department 
of  Energy,  etc.)  (PL  95-507:92  STAT.  1770).  Then,  in  1986, 
Congress  first  specified  that  a  "fair  proportion"  go  to  each 
industry  category  as  specified  by  the  Standard  Industrial 
Classification  (SIC)  code  (PL  99-500:100  STAT.  1783-147). 
Later,  they  decided  to  make  the  agencies'  goal  equal  to  five 
percent  of  the  total  combined  amounts  of  contracts  and 
subcontracts  in  each  of  the  four  following  categories: 

1.  procurement. 
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2.  research,  development,  testing,  and 

evaluation  (RDT&E), 

3.  military  construction, 

4.  operations  and  maintenance.  (29:100  STAT. 

1783-147) 

Studies  have  been  done  in  the  civilian  sector  regarding 
this  set-aside  percentage.  The  Center  for  Advanced 
Purchasing  Studies  (CAPS)  has  shown  through  statistical 
sampling  that  four  to  five  percent  of  the  supplier  base  for 
the  food  service  and  semiconductor  industries  are  minority 
owned  (23:3).  This  implies  that  for  some  industries  the 
goal  may  be  realistic. 

But  the  DOD  should  not  be  compared  to  these  civilian 
industries.  Merton  Peck  and  Frederic  Scherer  among  others 
have  pointed  out  that  DOD  purchasing  is  unlike  most 
commercial  activities  (21:57).  Further,  Jacques  Gansler,  in 
his  book  Affording  Defense,  claims  that  the  DOD  is  not  one 
big,  homogeneous  organization.  He  maintains  there  are 
dramatic  differences  between  the  various  sectors  of  defense, 
and  they  should  therefore  be  treated  differently.  (11:239- 
240)  In  addition.  Congress  realized  there  are  differences 
in  an  organization's  ability  to  meet  the  goal.  As  a  result, 
they  passed  the  Business  Development  Reform  Act  in  1988 
creating  different  goals  for  various  contract  categories  to 
determine  if  the  category  made  a  difference  (33:102  STAT. 
3890).  This,  combined  with  information  from  informal 


3 


interviews  with  SDB  representatives  and  research  performed 
by  other  researchers,  indicate  that  in  some  types  of  Air 
Force  contracting,  such  as  construction  and  routine  services 
(i.e,,  maintenance  service  and  food  services),  the  goal  may 
not  be  difficult  to  meet.  However,  in  other  areas  such  as 
research  and  development  and  electronic  production,  the 
goals  may  be  extremely  difficult  to  meet.  And,  if  this 
hypothesis  is  true,  that  is,  if  it  is  more  difficult  for 
some  contract  categories  to  meet  these  goals  than  others, 
then  perhaps  the  factors  prompting  these  differences  can  be 
determined . 

There  has  never  been  objective  research  conducted  to 
determine  whether  or  not  there  is  a  difference  based  on  the 
contracting  category  or  type  of  industry  contracted  with. 
Therefore,  this  study  was  performed  to  examine  the  ability 
of  different  AF  organizations  to  meet  the  specified  goal 
based  on  contracting  categories  and  type  of  industry. 

Research  Questions 

More  specifically,  the  study  attempted  to  answer  the 
following  question: 

Does  the  ability  of  AF  organizations  to  meet  the  five 

percent  goal  depend  on  contracting  categories? 

This  can  be  answered  through  the  following  specific  research 
questions : 

1.  Does  the  percentage  of  contract  dollars  going  to 

SDBs  meet  the  five  percent  goal ,  given  the  type  of 
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contracting  effort  and  the  specific  industry  or  product 
classification? 

2.  Does  the  percentage  of  contract  dollars  going  to 
SDBs  significantly  vary  between  category  of  contract 
effort  and  equipment/industry  classification? 

Scope  of  Study 

This  study  was  limited  to  investigating  the  abilities  of 
only  the  Air  Force  Systems  Command  (AFSC)  agencies  to  meet 
the  goal.  It  was  originally  intended  to  study  all  the  AF 
agencies  but  this  data  was  unavailable.  It  included  prime 
contracts  with  the  SDBs  and  SDB  contracts  with  the  Small 
Business  Administration  (SBA).  It  also  included  contracts 
with  Historically  Black  Colleges  and  Universities  (HBCU)  and 
Minority  Institutions  (MI).  It  did  not  include  contracts 
derived  as  a  result  of  the  Small  Business  Innovative 
Research  Program  because  this  method  of  contracting  was  not 
included  in  the  Congressional  mandate. 

When  analyzing  this  area,  one  must  realize  AF 
contracting  can  be  classified  in  many  ways.  In  fact,  there 
are  hundreds  of  categories  of  contracts.  These  categories 
are  not  necessarily  mutually  exclusive.  This  study  was 
primarily  interested  in  contracts  applicable  to  the  AF  but 
one  of  the  hurdles  was  to  determine  how  to  separate  the 
contracting  categories  into  applicable  groups. 

Fortunately,  the  data  system  used  for  the  information  in 
this  study  had  broken  down  all  the  AF  contacting  types  into 
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appropriate  groups.  The  first  group  is  based  on  the  Federal 
Supply  Classification  code  or  FSC  code.  This  breaks  the 
contracting  types  into  three  main  groups--(l)  Research 
Development,  Test,  and  Engineering  (RDTSE),  (2)  Other 
Services  and  Construction,  and  (3)  Supplies  and  Equipment. 
These  categories  are  appropriate  for  this  study  for  two 
reasons.  First,  they  break  the  contracting  types  into 
categories  which  Congress  stated  were  substitutable  for  the 
Standard  Industrial  Classification  (SIC)  codes  they  are 
interested  in  (38:1773).  Second,  the  categories  are 
stratified  sample  groups  which  may  have  significant 
differences  in  their  ability  to  meet  the  five  percent  goal. 
The  other  category  is  the  System  or  Equipment  Classification 
code  (SEC  code).  The  main  categories  are  aircraft,  aircraft 
engines,  missiles  and  rockets,  vehicles,  weapons, 
ammunition,  electronics,  and  other  equipment.  These 
categories  are  useful  because  of  the  interrelationship  to 
the  FSC  categories  and  their  similarity  to  the  different 
divisions  within  the  AFSC.  For  instance,  the  Aeronautical 
Systems  Division  (ASD)  deals  primarily  with  Aircraft  and 
Aircraft  Engines,  the  Space  Systems  Division  (SSD)  and 
Ballistic  Missile  Division  (BMD)  deal  primarily  with 
Missiles  and  Rockets,  the  Munitions  Systems  Division  (MSD) 
deals  primarily  with  the  weapons  and  munitions  and  the 
Electronic  Systems  Division  (ESD)  deals  with  the 
electronics.  The  interrelationship  of  these  categories  can 
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be  represented  as  shown  in  Figure  1. 
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Figure  1  Contract  Category  Relationship 


Limitations 

When  evaluating  the  results  and  conclusions  of  this 
research,  several  limitations  should  be  considered.  The 
AFSC  contracting  policy  may  be  unique  to  the  AF  and  other 
DOD  services.  Even  though  AFSC  contracting  follows  the 
federal  contracting  guidelines  set  forth  in  the  Federal 
Acquisition  Regulation  (FAR),  it  follows  more  specific 
guidelines  according  to  the  DOD,  AF,  and  AFSC  supplements  to 
the  FAR.  Other  organizations  may  divide  their  contracting 
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disciplines  differently,  especially  outside  the  DOD.  Within 
the  DOD  there  is  greater  standardization,  however,  some 
caution  should  be  exercised  when  associating  the  results  of 
this  research  to  other  AF,  service,  or  Federal  agencies. 

Definitions 

The  terms  frequently  used  throughout  this  proposal  are 
defined  as  follows: 

Small  Business:  A  business  which  is  independently  owned 
and  operated,  not  dominant  in  the  field  of  operation  in 
which  it  is  bidding,  and  meets  certain  size  requirements. 
This  depends  on  the  type  of  industry  the  business  is  in. 

For  example,  in  the  construction  industry,  small  business 
status  depends  on  the  average  annual  receipts  of  the 
business  whereas  in  the  manufacturing  industries  it  depends 
on  the  number  of  employees  of  the  business  and  its 
affiliates  (1:6-4). 

Small.  Disadvantaged  Business:  A  small  business  which 
meets  the  preceding  requirements  (depending  on  its  type  of 
industry),  is  at  least  51  percent  owned  by  one  or  more 
socially  and  economically  disadvantaged  individuals,  or  a 
publicly  owned  business  with  51  percent  or  more  stock  owned 
by  socially  and  economically  disadvantaged  individuals,  and 
is  managed  and  controlled  by  one  or  more  such  individuals 
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(12:150).  Women-owned  businesses,  for  the  purposes  of  this 
research,  are  not  included  in  this  category. 

Small  Business  Owned  and  Controlled  by  Socially  and 
Economically  Disadvantaged  Individuals:  The  same  as  Small, 
Disadvantaged  Businesses  (27:92  STAT.  1762). 

8(a)  Business:  A  subgroup  of  the  small,  disadvantaged 
businesses  which  are  participating  in  the  SBA's  8(a)  program 
(so  called  because  of  section  8(a)  of  the  Small  Business 
Act).  These  businesses  can  only  be  eligible  in  the  program 
for  a  fixed  period  of  time  (eight  years).  When  a  federal 
agency  contracts  with  these  businesses  they  actually 
subcontract  to  the  8(a)  business  through  the  SBA. 

There  are  many  advantages  for  participating  in  the  8(a) 
program.  First,  it  allows  the  business  to  get  exposure 
which  it  might  not  have  gotten  otherwise.  Second,  it  allows 
the  SBA  to  immediately  single  out  the  business  for 
consideration  if  the  SBA  thinks  they  are  eligible  for  the 
contract.  Third,  it  allows  the  businesses  special 
contracting  assistance  from  the  SBA. 

Thesis  Overview 

A  literature  review  to  review  the  background  and  some 
economic  theory  of  SDB  legislation  follows  this  chapter. 

The  methodology  for  the  research  is  described  in  detail  in 
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chapter  three.  Chapter  four  presents  the  findings  and 


analysis 

research 

research 


of  the  research.  Finally,  the  conclusions  of 
and  some  recommendations  resulting  from  the 
are  covered  in  the  last  chapter. 


the 
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Socio-economic  programs  in  general  and  SDB  programs  in 
particular  have  generated  considerable  attention  during  the 
last  two  decades.  As  a  result,  the  literature  on  this 
subject  is  extensive.  Potential  sources  of  information 
ranged  from  editorial  type  articles  to  empirical  research. 
This  literature  review  is  limited  to  background  information, 
legislation,  expert  testimony,  and  research  related  to  small 
business  and  SDB  programs,  especially  as  they  related  to  DOD 
and  AF  involvement  in  meeting  SDB  goals.  This  literature 
review  limited  its  scope  to  only  the  AF's  and  related 
organization's  participation  and  placed  greater  emphasis  on 
sources  which  substantiated  their  facts  as  opposed  to 
literature  providing  editorial  or  speculative  assessments. 
This  literature  review  consists  of  a  brief  legislative 
history  and  background  on  the  subject,  an  economic  analysis, 
arguments  for  and  against  the  legislation,  a  brief  analysis 
of  the  effectiveness  of  the  program,  and  a  discussion  of 
some  alternatives. 

Background 

When  reviewing  this  program,  one  must  keep  in  mind  that 


even  as  far  back  as  1865,  federal  acquisition  was  used  to 


implement  socio-political  programs.  At  that  time  Congress 
mandated  that  only  American  bunting  could  be  purchased  and 
American  labor  and  materials  were  preferred  for  public 
improvement  contracts  (26:41).  However,  the  legislation 
which  pertains  to  this  study  dates  back  to  1953  when 
Congress  created  the  Small  Business  Administration  (SBA) 
with  the  passage  of  the  Small  Business  Act.  This 
legislation  attempted  to  help  those  businesses  which  had  a 
small  chance  of  becoming  competitive  because  of  their  size. 

In  1978,  Congress  passed  PL  95-507,  Amendment  to  the 
Small  Business  Act  of  1953.  The  intention  was  to  start  a 
socio-economic  program  which  would  make  up  for  past 
discriminatory  practices  against  minority  groups  such  as 
women,  blacks,  and  hispanics,  among  others.  Previously, 
most  of  the  small  business  legislation  emphasized  small 
businesses,  not  necessarily  small,  disadvantaged  businesses. 
This  law  was  also  supposed  to  allow  these  groups,  which 
traditionally  own  a  disproportionately  small  portion  of  the 
businesses  in  the  US,  to  have  a  chance  to  become  viable 
competitors  in  the  US's  free  enterprise  system.  The  law 
itself  states: 

...it  is  therefore,  the  purpose  of  section  8(a)  to- 

'(A)  foster  business  ownership  by  individuals  who  are 
both  socially  and  economically  disadvantaged, 

'(B)  promote  the  competitive  viability  of  such  firms  by 
providing  such  available  contract,  financial,  technical, 
and  management  assistance  as  may  be  necessary;  and 

'(C)  clarify  and  expand  the  program  for  the  procurement 
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by  the  United  States  of  articles,  equipment,  supplies, 
services,  materials,  and  construction  work  from  small 
business  concerns  owned  by  socially  and  economically 
disadvantaged  individuals.  (27:92  STAT.  1760-1761) 

The  US  senate  further  clarified  the  law  and  reported  the 

foil  owing : 

The  purpose  of  the  8(a)  program  is  to  foster  the 
development  of  socially  and  economically  disadvantaged 
businesses  and  to  promote  the  competitive  viability  of 
these  businesses  by  providing  necessary  contract, 
financial,  technical  and  management  assistance.  (3:5600) 

Some  chief  provisions  of  the  act  were:  (1)  it  included 

socially  disadvantaged  individuals  into  the  program,  (2)  it 

allowed  the  SBA  to  contract  with  the  SDB's,  (3)  it  increased 

the  dollar  amount  going  to  Small  Businesses  through 

subcontracts  with  federal  agencies,  and  (4)  it  established 

goals  which  could  be  set  by  the  heads  of  federal  agencies 

and  the  administration  (35:5603,  27:92  STAT.  1770). 

Unfortunately,  in  the  article  "Socioeconomic  Contract  Goal 

Setting  Within  the  Department  of  Defense:  Promises  Still 

Unfulfilled",  Dennis  Black,  demonstrates  that  neither  the 

implementation  of  the  law  nor  the  five  percent  goal 

accomplished  the  intended  results  (3:67-82).  Arthur  King 

also  confirmed  this  in  his  doctoral  thesis  (10:16). 

In  1982,  Congress  passed  the  Small  Business  Innovation 

Development  Act,  which  established  the  Small  Business 

Innovation  Research  Program  (SBIR).  This  legislation  aimed 

at  R&D  business  development  had  four  objectives: 

1.  stimulate  technological  innovations, 

2.  use  small  businesses  to  meet  federal  R&D  needs. 
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3.  foster  and  encourage  participation  by  minority  and 
disadvantaged  persons  in  technological  innovation, 

4.  increase  private  sector  commercialization 
innovations  derived  from  federal  RSD.  (28:196  STAT. 

217) 

However,  when  Congress  reviewed  the  original  proposal,  the 
DOD,  NASA,  the  DOE,  and  the  SBA  were  against  the  one  percent 
set-aside  for  small  businesses  because  it  would  reduce  their 
flexibility  (36:521).  Nevertheless,  Congress  passed  the 
proposal  anyway  (36:523). 

Then,  in  1986,  Congress  took  major  steps  towards 
improving  the  effectiveness  of  PL95-507.  Through  the 
Defense  Appropriations  Act  and  the  Continuing  Appropriations 
Acts  for  Fiscal  Year  1987  (PL99-661,  PL99-500,  and  PL99-591 
respectively).  Congress  specified  that  a  ’’fair  proportion" 
of  the  federal  agency's  money  go  to  each  industry  category 
as  specified  by  the  four-digit  SIC  (Standardized  Industrial 
Classification)  code  (31:100  STAT.  3927).  Strangely  enough, 
the  joint  committee  reviewing  the  resolution  had  no  comments 
on  this  portion.  However,  later,  in  the  Defense 
Appropriations  Act,  the  House  wanted  to  mandate  ten  percent 
of  the  DOD's  dollars  in  each  of  the  following  four  areas  to 
set  aside  for  SDBs,  HBCUs  (Historically  Black  Colleges  and 
Universities),  and  Mis  (Minority  Institutions):  (1) 
Procurement,  (2)  RDTSE,  (3)  military  construction,  and  (4) 
operations  and  maintenance.  At  this  point  SDBs,  HBCUs,  and 
Mis  were  labeled  Section  1207(a)  entities  because  they  were 
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specified  as  such  in  Section  1207(a)  of  the  resolution. 
However,  the  Senate  softened  the  legislation  by  dropping  the 
mandate,  making  it  a  goal  instead,  and  cutting  the  ten 
percent  back  to  five  percent.  This  is  where  the  DOD  gets  it 
five  percent  goal  today.  At  the  same  time,  the  Senate 
agreed  to  allow  less  than  full  and  open  competition  for 
competing  the  set-asides  and  allowed  the  contracts  to  cost 
up  to  ten  percent  more  than  the  fair  market  value  of  the 
contract  (31:6583-6584). 

The  next  year,  in  the  Defense  Appropriations  Act  for 
Fiscal  Year  (FY)  1988,  Congress  mandated  that  the  DOD  make 
substantial  progress  for  awarding  contracts  to  section 
1207(a)  entities  (32:101  STAT.  1126).  The  House  also 
clarified  that  federal  agencies  could  group  the  SIC  codes  to 
establish  their  annual  goals  (38:1772).  This  was  a  result 
of  an  extreme  difference  in  set-aside  amounts  depending  on 
which  SIC  codes  were  used  for  refuse  collection  (32:101 
STAT.  1126).  At  the  same  time  they  realized  existing 
federal  classification  codes  such  as  the  Federal  Supply 
Classification  (FSC)  code  were  similar  to,  and  could 
accomplish  the  same  objectives  as,  the  SIC  codes.  Therefore, 
they  also  allowed  a  correlation  between  Federal  Procurement 
Data  System  (FPDS)  codes  and  SIC  codes  (38:1773). 

In  1988,  Congress  passed  the  Business  Development  Reform 
Act,  the  purpose  of  which  was  to  demonstrate: 

(1)  the  competitive  capabilities  of  small  business  firms 

in  certain  industry  categories  will  enable  them  to 
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successfully  compete  on  an  unrestricted  basis  for 
federal  contracting  opportunities, 

(2)  the  use  of  targeted  goaling  and  management 
techniques  by  procuring  agencies  in  conjunction  with  the 
Small  Business  Administration,  can  expand  small  business 
participation  in  Federal  contracting  opportunities  which 
have  been  historically  low,  despite  adequate  numbers  of 
qualified  small  business  contractors  in  the  economy, 

(3)  expanded  use  of  full  and  open  competition,  adversely 
affects  small  business  participation  in  certain  industry 
categories,  taking  into  consideration  the  numerical 
dominance  of  small  firms,  the  size  and  scope  of  most 
contracting  opportunities,  and  the  competitive 
capabilities  of  small  firms.  (33:102  STAT.  3890). 

Basically,  this  legislation  enhanced  small  business  (note: 

not  SDBs)  participation  goals  to  40  percent  in  groups  with 

historically  low  rates  of  small  business  participation  and 

large  numbers  of  qualified  firms  (33:102  STAT.  3890-3891). 

The  groups  of  interest  are;  (1)  construction  (SIC  codes 

15XX,  16XX,  and  17XX),  (2)  refuse  systems  and  related 

services  (4212,  4953),  (3)  architectural  and  engineering 

services  (7389,  8711,  8712),  (4)  non-nuclear  ship  repair 

(SIC  code  not  specified),  and  dredging  (SIC  code  not 

specified)  (37:5478).  The  requirements  are  the  same  for  all 

of  the  categories  except  dredging  which  had  a  phase-in 

period  of  participation  goals.  The  previous  goal  for  small 

businesses  was  20  percent.  The  interesting  part  here  is  the 

fact  this  legislation  was  a  result  of  a  survey  done  among 

8(a)  program  "graduates"  (businesses  which  had  finished  the 

program  after  eight  years).  This  survey  showed  two  main 

problems  (37:5404,  5406).  The  first  problem  was  there  were 
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shortcomings  in  the  day-to-day  operations  and  the  long  term 
effects  of  the  program.  The  second  problem  was  the  program 
was  viewed  more  as  a  contracts  program,  not  as  an 
opportunity  to  develop  the  business.  The  second  problem 
conflicts  with  one  of  the  main  purposes  of  the  program. 

Most  recently,  in  the  Defense  Appropriations  Act  for  FY 
1990,  Congress  mandated  the  DOD  to  increase  resources  and 
the  number  of  personnel  to  help  promote  and  assist  SDBs  to 
gain  an  equal  chance  to  provide  commodities  and  services  and 
allowed  work  done  on  indian  lands  to  count  as  part  of  the 
goal  (34:103  STAT .  1129). 

Two  sources  suggested  reasons  tor  DOD  involvement  in 
this  program.  Dale  McNabb  et  al . ,  in  their  article  "On  the 
Utilization  and  Degradation  ot  LCD  Acquisition  System 

for  Socio-Economic  Policy  Implementation,"  state  that  the 
DOD  must  be  used  since  their  portion  of  governmental  outlays 
is  so  large  (18:22).  Also,  B.R.  Gamble’s  research  study  of 
the  AF's  participation  in  the  8(a)  program  explains  that  the 
DOD  must  be  used  because  it  is  the  "'big  spender'"  of 
federal  agencies  and  goes  on  to  point  out  that  the  DOD  was 
the  largest  single  item  in  federal  outlays  until  1974 
(10:12) . 

Realizing  this  fact,  the  AF  started  a  program  before  any 
of  these  laws  were  enacted.  Major  General  Stansberry  of  the 
Air  Force  Systems  Command  (AFSC)  explained  that  the  AFSC 
encouraged  the  use  of  Minority  Business  Enterprises  (MBE) 
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one  year  before  PL  95-507  was  passed.  He  asserted  that  the 
entire  command  was  instructed  to  contract  with  MBEs  as  much 
as  possible  (25:40-43).  Other  examples  of  Air  Force  support 
for  the  program  include  a  letter  from  the  Commander  in  Chief 
of  the  Military  Airlift  Command  and  yearly  statistics 
showing  Air  Force  support  (lOratch  1; 

3:67-82) . 

Based  on  the  previous  section,  there  has  been  much 
analysis  on  the  social  aspect  of  this  legislation.  However, 
how  does  this  legislation  affect  the  players  economically? 

An  Economic  Analysis 

In  this  section  this  analysis  will  discuss  the  economic 
aspects  of  this  program  as  it  relates  to  the  DOD.  The 
reader  should  bear  in  mind,  though,  that  this  program 
applies  to  all  federal  agencies.  The  analysis  will  address 
the  theory  behind  socio-economic  programs  such  as  the  SDB 
set-aside  program,  the  DOD’s  economic  market  and  the 
program's  effect  on  it,  and  some  economic  obstacles  the  SDBs 
have  in  the  DOD  market. 

Theory  of  Socio-Economic  Programs.  Observations  on  the 
role  of  government  and  socio-economic  programs  in  a 
capitalist  economy  such  as  the  US's  have  been  made  since  its 
founding.  James  Madison,  one  of  the  founding  fathers  of  our 
country,  believed  a  capitalistic  system  would  lead  to  a  need 
for  regulation  because  of  basic  human  nature: 
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Those  who  hold  and  those  who  are  without  property  have 
ever  formed  distinct  interests  in  society.  ...  A 
landed  interest,  a  manufacturing  interest,  a  mercantile 
interest,  a  moneyed  interest,  with  many  lessor 
interests,  grew  up  of  necessity  in  civilized  nations, 
and  divide  themselves  into  different  classes,  actuated 
by  different  sentiments  and  views.  The  regulation  of 
these  various  and  interfering  interests  involves  the 
spirit  of  party  and  faction  in  the  necessary  and 
ordinary  operations  of  government.  (2:43-44) 

More  recently,  Edward  Chamberlain,  in  his  essay  "Product 

Heterogeneity  and  Public  Policy,”  also  remarked:  "Public 

policy  must  be  presvimed  to  seek  in  some  sense  the  general 

welfare,  and  hence  in  the  economic  sphere  it  implies  a 

welfare  economics  (4:236).”  Chamberlain  goes  on  to  point 

out  we  wish  to  approach  a  workable  competition  in  society 

through  these  welfare  programs  (4:236-237).  If  it  can  be 

assumed  these  programs  improve  the  spirit  of  the  people, 

Gansler  also  infers  the  importance  of  these  programs  when  he 

states  "the  social  unity  and  political  resolve  ...  of  the 

public  to  deploy  and  support  forces  .  .  .is  crucial  (11:19). 

These  programs  are  also  important  because  they  help  drive 

the  system  to  satisfy  the  three  necessary  conditions,  which, 

according  to  Edwin  Mansfield,  in  his  book  Microeconomics . 

Theory  and  Applications,  allow  for  optimal  resource 

allocation  in  an  economic  system:  (1)  the  marginal  rate  of 

substitution  between  any  two  commodities  must  be  the  same 

for  any  two  consumers,  (2)  the  marginal  rate  of  technical 

substitution  between  any  two  inputs  must  be  the  same  for  any 

pair  of  producers,  and  (3)  the  marginal  rate  of  substitution 
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between  any  two  commodities  must  be  the  same  as  the  marginal 
rate  of  product  transformation  between  two  commodities  for 
any  producer  (17:470-472). 

Based  on  these  insights,  it  is  possible  to  conclude  that 
in  our  free  enterprise  society,  not  only  do  we  need  these 
socio-economic  programs  for  people  in  the  society,  but  for 
the  economic  system  as  well.  Keeping  this  in  mind,  how  does 
the  DOD  and  the  SDB  set-aside  program  solve  this  need? 

The  POD  and  SDB  Set-Asideg.  The  program  is  designed  to 
make  up  for  past  discriminatory  practices  in  the  US  and  to 
increase  competition  in  the  free  enterprise  system.  As 
Madison  pointed  out,  there  are,  were,  and  will  continue  to 
be  different  factions  of  wealth  in  this  country.  Gordon 
Bjork  points  out  in  his  book.  Private  Enterprise  and  Public 
Interest:  The  Development  of  American  Capitalism,  that 
minorities,  especially  blacks,  have  not  been  equally  treated 
in  the  US  (2:221-225).  Therefore,  in  the  US’s  interest  it 
is  necessary  to  use  socio-economic  programs  to  make  up  for 
these . 

As  stated  earlier,  the  DOD  is  an  obvious  choice  for  the 
implementation  of  this  program  because  of  its  immense 
portion  of  the  federal  budget.  In  1988,  the  defense  portion 
of  the  budget  was  25  percent  (11:79).  Kow  does  this  program 
interact  with  the  DOD  economic  market?  To  determine  this, 
the  DOD  market  structure  must  first  be  defined. 

The  defense  industry  currently  has  relatively  few  actual 
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suppliers.  There  are  thousands  of  small  firms  in  the 
market,  but  their  impact  is  minimal  since  they  are  mostly 
subcontractors  to  the  few  large  firms  the  DOD  deals  with. 
For  example,  consider  the  high  concentration  of  business 
given  to  the  top  four  defense  firms  in  the  various  sectors 
of  the  defense  market  as  shown  in  the  Table  1. 


Table  1 

Percentage  of  Defense  Market 
for  Four  Largest  Defense  Contractors  (11:246) 


Portion  of  Defense  Market  Percentage 

Surveillance  and  Satellites  100 
Nuclear  Submarines  99 
Space  Boosters  97 
Fighter  Aircraft  97 
Attack  Aircraft  97 
Missile  Inertial  Guidance  Systems  97 
Aircraft  Inertial  Navigation  Systems  96 
Missile  Reentry  Vehicles  95 
Aircraft  Fire  Control  Systems  95 
Transport  and  Tanker  Aircraft  94 
Helicopters  93 
Jet  Aircraft  Engines  93 


Although  this  does  not  cover  the  full  spectrum  of  the  DOD's 
business,  it  substantiates  the  point  that  there  are  only  a 
few  substantial  suppliers  in  the  DOD  market.  Based  on  this 
information,  a  single  buyer  and  few  suppliers,  the  DOD 
market  structure  could  be  classified  as  a  bilateral 
monopoly,  a  monopsony,  or  an  oligopoly.  Some  even  say  the 
DOD  market  is  so  unique  with  respect  to  weapon  systems  it 
has  no  market  structure  at  all  (21:55).  The  following 
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analyzes  these  possible  market  structures. 


Jacques  Gansler  points  out  in  his  book.  Affording 
Defense .  that  for  any  given  weapon  system,  there  is  only  one 
buyer  and  one  or  two  suppliers  (11:246).  This  appears  to  be 
a  possible  bilateral  monopoly  since  a  bilateral  monopoly 
consists  of  one  buyer  and  one  seller.  However,  according  to 
Frederic  Scherer,  in  his  book  The  Weapons  Acquisition 
Process:  Economic  Incentives,  two  firms  may  be  enough  to 
keep  it  from  being  a  bilateral  monopoly.  He  states  that  for 
optimal  incentive  effects  the  number  of  competitors  only 
needs  to  be  large  enough  that  there  is  a  real  threat  of 
cancellation  (or  losing)  the  contract  (24:48).  According  to 
this  view,  as  long  as  there  is  enough  competition  to 
preclude  a  monopoly  with  respect  to  suppliers,  a  bilateral 
monopoly  may  not  take  place. 

According  to  Edwin  Mansfield,  the  defense  industry  is 
an  oligopoly  (17:337).  However,  with  a  single  buyer  (the 
federal  government)  a  true  oligopoly  cannot  exist  since  it 
requires  a  multiple  buyer  market  (a  demand  curve  which 
cannot  be  changed  or  influenced  by  one  individual  or  firm). 
The  government  certainly  controls  the  demand  curve  in  the 
defense  industry.  Therefore,  would  be  oligopolist  firms  in 
the  defense  industry  cannot  control  the  market  as  they  would 
in  a  true  oligopoly. 

A  monopsony  is  a  situation  in  which  there  is  a  single 
buyer.  Although  this  is  the  case  in  the  defense  industry  it 
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may  not  be  a  true  monopsony  because  the  sellers  may  be  able 
to  use  an  alternate  market  to  sell  their  goods.  Some  of  the 
larger,  more  diversified  sellers  can  sell  their  services  and 
manufacturing  skills  to  the  civilian  industry  market,  so 
they  are  not  dependent  on  the  single  buyer  (the  government). 

On  the  extreme,  Merton  Peck  and  Frederic  Scherer  believe 
no  market  structure  exists  at  all  with  respect  to  weapon 
systems  in  the  defense  industry.  They  claim  that  in  a 
market:  (1)  the  seller  initiates  the  decision  to  produce  and 
the  buyer  then  decides  to  purchase,  (2)  prices  are 
determined  by  competition,  not  cost  and  fair  return,  and  (3) 
there  are  decentralized  decisions  on  what  to  produce  and 
price,  and  since  the  defense  industry  doesn't  meet  these 
criteria  it  has  no  market  structure  in  the  traditional  sense 
(21:55-57).  According  to  Mansfield,  a  market  is  a  "group  of 
firms  and  individuals  in  touch  with  others  in  order  to  buy 
or  sell  some  goods  (8:20)."  Although  this  definition  is 
generic  in  nature,  it  still  points  out  that  with  buyers  and 
sellers  involved  we  have  a  market.  And,  with  a  market  we 
must  have  some  structure  since  we  have  interaction  between 
the  buyers  and  sellers.  Albeit  it  may  not  be  well  defined 
and  accepted  structure,  it  is  one  nonetheless.  In  addition, 
this  researcher  disagrees  with  Peck  and  Scherer's  analysis 
because  all  of  their  elements  of  a  market  can  and  do  take 
place  in  the  DOD  industry.  Only  in  the  case  of  major 
weapons  acquisition  do  all  the  elements  not  exist. 
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If  the  defense  industry  isn't  a  bilateral  monopoly, 
oligopoly,  monopsony,  or  non-market  structure,  what  is  it? 

It  is  a  combination  of  oligopoly  and  monopsony  which  creates 
a  unique  market  structure  ruled  by  a  single  buyer  whose 
demand  supports  a  supply  by  a  limited  number  of  firms.  As 
such,  the  demand  curve  is  relatively  inelastic,  since  price 
is  not  really  much  of  a  driver  in  determining  what  (or  how 
many)  of  an  item  is  bought.  And,  the  supply  curve  is  also 
relatively  inelastic  for  the  same  reasons  in  addition  to  the 
fact  there  are  no  close  substitute  goods. 

With  this  type  of  a  market  structure,  what  are  the 
effects  of  the  SDB  set-aside  program?  Three  of  the  possible 
effects  are  lower  costs,  increased  competition  and  a  reduced 
chance  of  collusion. 

One  of  the  economic  goals  of  the  program  is  to  increase 
the  competition  in  the  defense  industry  and  thus  lower 
costs.  With  basic  economic  theories,  one  can  show  that  as 
one  increases  competition,  in  general,  costs  are  lowered 
(17:324).  This  is  due  to  the  price  of  goods  approaching  the 
optimum  price  where  supply  equals  demand  rather  than  an 
optimum  price  established  by  a  firm  or  firms  where  marginal 
costs  equal  marginal  revenue  (as  in  the  case  of  a  monopoly). 
This  is  a  desired  additional  economical  benefit  of  the  SDB 
set-aside  program. 

Another  hoped-for  benefit  of  additional  competition 
through  the  SDB  set-aside  program  is  the  reduced  chance  of 
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collusion  taking  place.  Earlier,  the  analysis  showed  that 
as  long  as  there  are  two  suppliers,  and  the  belief  that  one 
may  lose  the  contract,  there  is  sufficient  competition. 

This  assumes  no  collusion  takes  place  or  cartels  are  formed. 
Obviously,  an  increase  in  the  number  of  firms  in  any  one 
industry  will  lessen  the  chance  of  collusion  taking  place  or 
cartels  forming. 

But,  these  benefits  may  not  occur  in  this  case.  Walter 
Culver,  in  his  article  "The  Federal  Small  Business  Program: 
Does  it  Really  Serve  the  National  Interest?",  implies  that 
the  SDB  set-aside  program  hurts  competition  rather  than 
helps  it  because  the  program  encourages  the  SDBs  to  stay 
small  through  its  regulations  (5:24-25).  This  was  also 
verified  in  Senate  Report  Number  100-394,  during  the  review 
of  Public  Law  100-656,  the  Business  Development  Reform  Act 
(33:5406).  And,  in  order  to  enter  the  competitive  markets 
in  the  defense  industry,  the  SDBs  will  have  to  invest 
substantially  in  the  capital  to  acquire  and  maintain 
equipment  in  order  to  become  viable  competitors.  This  cost 
could  be  extraordinary  and  would  likely  be  paid  for  by  the 
government  through  increased  prices  on  contracts.  In 
addition,  Gansler  points  out  these  SDBs  usually  have  cost 
overruns,  late  deliveries,  and  then  default  on  contracts 
forcing  them  to  ask  the  government  for  relief  (11:262). 

Another  factor  to  consider  is  the  impact  of  variable 
defense  funding  by  Congress.  If,  during  a  time  of  high 
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defense  funding,  the  industrial  base  is  increased  by  adding 
these  highly  specialized  SDBs,  who  will  be  hurt  the  most 
during  a  period  of  reduced  funding?  It  will  probably  not  be 
the  relatively  diversified  major  contractors  but  the  highly 
specialized  SDBs,  which  could  cause  an  increased  burden  on 
the  welfare  system  through  unemployment. 

In  addition  to  all  of  these,  the  contracting  agencies 
are  allowed  to  pay  up  to  ten  percent  more  than  a  fair  market 
price,  thus  possibly  driving  the  prices  up  even  further. 
Therefore,  the  program  does  not  necessarily  increase 
competition  and  reduce  costs. 

Obstacles  for  SDBs  in  the  POD  Market.  Since  the  defense 
industry  is  a  pseudo-oligopolist  market,  the  barriers  to 
enter  an  oligopolist  market  need  to  be  considered. 

According  to  Mansfield  there  are  three  main  barriers; 

1.  size  of  the  market  relative  to  the  size  of  the  firm, 

2.  building  and  maintaining  a  large,  complicated,  and 

expensive  plant,  and 

3.  unavailability  of  natural  resources  (17:354). 

The  second  point  has  already  been  discussed.  The  third 
point  is  relative  in  terms  of  labor  as  a  natural  resource. 
The  first  point  is  more  important.  The  firms  must  be  able 
to  cover  all  their  coats.  Once  enough  firms  exist  to  drop 
the  individual  demand  curve  below  the  average  cost  curve,  an 
effective  barrier  to  entry  exists  (17:354). 
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There  are  other,  more  specific  barriers  to  entry, 
though.  Gansler  discusses  the  following: 

1.  brand  loyalty  by  services, 

2.  need  for  high  levels  of  engineering  and  scientific 

capabilities  (a  more  specific  case  of  Mansfield's  third 

barrier) , 

3.  need  for  large  cost  reserves, 

4.  specialized  reporting  requirements, 

5.  required  knowledge  of  the  detailed  federal 

regulation, 

6.  security  clearances,  and 

7.  political  considerations.  (11:246) 

All  of  these  barriers  cause  additional  difficulty  for  the 
SDBs  to  enter  the  market.  In  addition,  Gansler  points  out 
barriers  to  diversification,  leading  to  problems  discussed 
earlier  with  respect  to  funding  levels  (11:246). 

Conclusion  ' /en  though  Congress  has  implemented 
socio-econonn.-.  legislation  via  the  SDB  set-aside  programs  to 
help  the  economic  situation  of  these  SDBs,  and  these 
socio-economic  programs  are  necessary,  the  SDB  set-aside 
program  does  not  have  sound  economic  logic.  First,  because 
of  its  unique  monopsonistic-oligopol istic  structure,  the  DOD 
is  not  a  market  in  which  set-asides  are  necessarily 
advantageous.  The  SDBs  may  not  lower  costs,  increase 
competition  or  reduce  the  possibility  of  col lusion--al 1 
desired  benefits  of  the  program.  Second,  there  are  numerous 
and  large  barriers  to  enter  the  defense  market,  not  only 
those  usually  identified  with  an  oligopolistic  market  but 
additional  ones  specifically  associated  with  the  defense 
industry.  This  makes  it  even  harder  for  the  program  to 
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achieve  its  economic  goals.  In  addition  to  these  defense 
related  obstacles,  there  are  other,  more  general, 
impediments  as  well. 

Impediments  to  SDB  Use 

One  study  revealed  that  there  are  a  number  of 
impediments  to  using  SDBs.  In  the  report  entitled 
Pur chasinq_rTQm.  Small  Minority-Owned  Firms:  Corporate 
Problems .  Marc  Dol linger  and  Catherine  Daily  show  that  there 
are  four  chief  impediments  to  MBE  (minority  business 
enterprise)  use.  They  are  (in  order  of  importance):  (1) 
business  uncertainty,  (2)  small  numbers,  (3)  opportunism, 
and  (4)  atmosphere  (7:13).  The  results  of  the  study  also 
show  that  the  following  four  impediments  were  not  problems 
when  dealing  with  MBEs:  (1)  impacted  information,  (2) 
resource  dependence,  (3)  complexity,  and  (4)  product 
uncertainty  (7:13).  The  three  types  of  firms  surveyed  for 
this  information  were;  large  corporations,  small  businesses, 
and  MBEs.  Although  the  combined  sample  showed  that  the 
number  one  problem  with  conducting  business  with  MBEs  was 
due  to  small  numbers,  the  ratings  by  individual  group  were 
different.  To  the  corporations,  the  largest  impediment  was 
complexity  (7:14).  They  also  scored  higher  than  the  other 
groups  in  the  small  numbers,  business  uncertainty,  and 
production  uncertainty  categories  (7:14).  However,  the  MBEs 
thought  complexity,  opportunism,  atmosphere,  and  impacted 
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information  were  larger  factors  (7:14).  As  a  result,  not 
only  are  there  DOD  specific  impediments  to  SDB  use  but 
general  impediments  met  by  civilian  businesses  as  well. 

Advantaqes/Disadvantaqes 

The  literature  revealed  some  advantages  and 
disadvantages  of  the  SDB  program.  The  advantages  of  the 
program  are  innovation,  an  increase  of  the  industrial  base, 
and  the  ability  to  make  up  for  past  discrimination.  Some  of 
the  literature  showed  that  innovation  can  be  an  especially 
good  benefit  of  the  program.  Steve  Edelmann's  article, 

"What  Makes  America  Great,"  advocates  more  involvement  of 
8(a)s  in  DOD  research  and  development  (R&D)  because  25 
percent  of  all  PhD  diplomas  awarded  to  scientists,  engineers 
and  such  were  awarded  to  members  of  minority  groups  that 
could  qualify  as  8(a)  business  owners  (8:32).  In  addition. 
Dr.  Yale  Lubkin  points  out  several  historical  examples  of 
successful  companies  which  started  as  small,  innovative 
companies.  One  example  he  gives  is  Edwin  Land  joining  the 
Haloid  company  to  form  Xerox  (15:34). 

Likewise,  another  advantage  of  the  program  is  to 
increase  the  industrial  base.  Senator  Alan  Dixon,  D-IL, 
states:  "The  security  of  the  United  States  is  dependent  on 
its  industrial  base  however,  and  small  business  must  be 
encouraged  to  participate  if  we  are  serious  about 
maintaining  our  technological  advantage,  whether  in  peace  or 
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war  (6:14)."  And,  as  one  would  hope,  since  it  is  the 
ultimate  goal  of  the  program,  the  program  can  possibly  make 
up  for  past  discrimination.  Arthur  King,  in  his  study  on 
the  8{a)  program  and  the  AF,  concluded  "the  8(a)  Program  has 
been  somewhat  effective  in  alleviating  one  aspect  [past 
discrimination]  of  the  economic  underdevelopment  problem  of 
the  United  States  minority  group  sector"  and  has  helped  a 
limited  number  of  minority  group  businesses  (10:16). 

But,  the  research  shows  there  are  some  disadvantages, 
too.  The  chief  disadvantages  are  increased  prices, 
increased  completion  time  of  acquisitions,  and  increased 
costs.  John  Magnotti's  study  of  the  SBA's  pilot  program 
with  8(a)s  that: 

Congress  is  being  used  to  generate  legislation  that  runs 
from  ineffective  to  damaging  to  all  parties  concerned: 
the  minority  business  establishment,  the  contractual 
programs  involved,  and  the  majority  contractors  caught 
up  in  contractual  actions  'reserved  by  the  SBA. '  (16:64) 

Comparable  to  this  is  Timothy  Bates'  research  which  shows 

government  contracting  officers  contend  buying  from  8(a) 

firms  usually  produced  low  performance  and  late  deliveries 

(14:38).  In  addition  to  this,  extra  costs  are  incurred  from 

the  program. 

The  extra  costs  incurred  include  the  basic  cost  of 
running  the  program.  Congress  did  an  estimate  during  the 
analysis  of  PL  95-507  to  see  how  much  the  administration  of 
the  program  would  cost.  The  total  cost  was  estimated  to  be 
$35.6  million  (13:5615).  Plus,  the  DOD  is  authorized  to  pay 
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up  to  ten  percent  more  than  the  fair  market  price  for 
contracts  set  aside  for  an  8(a)  business.  Thirdly,  Black 
points  out  that  the  costs  of  hiring  an  8(a)  business  are 
greater  than  any  contractor  might  receive  for  subcontracting 
to  the  8(a)  business  (3:78). 

Effectiveness 

There  is  much  information  pointing  out  the 
ineffectiveness  of  the  program.  Black  points  out  several 
examples  of  the  program's  inability  to  accomplish  its  goals. 
He  shows  there  has  been  no  significant  increase  in  the  share 
of  dollars  going  to  the  8(a)s  (3:80).  He  also  states  "there 
is  no  evidence  to  date  that  such  use  of  an  agency's 
contracting  process  .  .  .  is  an  effective  means  of  reducing 
socioeconomic  discrimination  in  society  (3:80)."  And,  he 
claims  the  program  has  not  significantly  increased  the  share 
of  federal  dollars  to  8(a)  businesses  (3:81).  Lorette  et 
al.,  propose  the  progri...!  is  actually  a  hinderance  to  the 
companies.  They  claim  too  much  effort  has  been  directed 
towards  obtaining  contracts  and  not  towards  "teaching  the 
skills  required  to  complete  the  contract  and  to  stay  in 
business  (14:39)."  Walter  Culver  also  points  out  that  the 
program  hurts  competition  rather  than  helping  it  and  the 
program  encourages  small  businesses  to  stay  small  (5:24-25). 

Based  on  the  previous  discussion,  it  is  evident  there  is 
much  effort  towards  increasing  SDB's  share  of  the  business 
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in  this  country,  however,  it  is  not  effective  effort.  There 
must  be  some  alternatives  to  make  the  system  work  better. 

Alternatives 

Many  of  the  articles  had  proposed  alternatives. 

Increased  incentives  rather  than  mandated  goals, 
subcontracting  rather  than  set-asides,  and  reserving 
specific  amounts  in  specific  areas  of  the  federal  business 
for  8(a)s  are  three  suggested  alternatives.  Black,  Culver 
and  Magnotti  believe  giving  government  agencies  and 
contractors  more  incentive  to  contract  with  8(a)s  will  be 
more  effective  (3:81;  5:25;  16:64).  Edelmann  believes  the 
government  should  concentrate  on  increasing  8(a) 's  share  of 
subcontracting  while  McNabb  believes  the  government  should 
reserve  a  certain  amount  of  the  contracting  fcr  the  8(a)s 
rather  than  a  five  percent  requirement  across  the  board 
(8:32;  18:26). 

Dol linger  and  Daily  listed  nine  recommended  activities 
for  aiding  MBEs .  They  are  (in  order  of  importance):  (1) 
search  for  corporations,  (2)  monitoring  of  MBEs,  (3) 
managerial  assistance  for  MBEs,  (4)  searching  for  MBEs,  (5) 
cultural  interaction,  (6)  monitoring  of  corporations,  (7) 
internalization,  (8)  technical  assistance,  and  (9)  financial 
assistance  (7:13).  It  is  interesting  to  note,  especially 
since  the  MBEs  participated  in  this  survey,  that  financial 
assistance  is  at  the  bottom.  The  most  recommended 
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activities  are  administrative  functions  which  the  SBA  should 


do . 

Cciig.l.visign 

Clearly,  based  on  the  literature  reviewed,  this  program 
to  help  8(a)  businesses,  derived  from  Public  Laws  95-507, 
99-601,  and  100-180,  is  not  as  effective  as  it  could  be. 
This  study  performed  a  quantifiable  analysis  to  determine 
whether  there  are  certain  contracting  categories  and 
equipment/industrial  classifications  which  impact  the  AF's 
a,^ility  to  meet  the  five  percent  goal.  If  the  analysis 
shows  the  program  is  not  accomplishing  its  goals,  then  some 
better  alternatives,  or  possibly  a  different  program 
altogether,  should  be  implemented  which  will  accomplish  the 
goals . 
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III.  Methodology 


Introduction 

The  focus  of  this  research  was  to  assess  the  ability  of 
Air  Force  organizations  to  meet  the  five  percent  Small  and 
Disadvantaged  Business  (SDB)  goal  established  by  Congress 
for  Federal  Agencies.  Specifically,  it  sought  to  determine 
whether  the  type  of  contracting  effort  and  the  type  of 
industry  or  product  involved  impacted  the  organization's 
ability  to  meet  the  five  percent  goal.  In  order  to  do  this, 
the  following  two  research  questions  were  examined: 

1.  Does  the  percentage  of  contract  dollars  going  to 
SDBs  meet  the  five  percent  goal,  given  the  type  of 
contracting  effort  and  the  specific  industry  or  product 
classification? 

2.  Does  the  percentage  of  contract  dollars  going  to 
SDBs  significantly  vary  between  category  of  contracting 
effort  and  equipment/industry  classification? 

This  chapter  presents  the  methodology  used  to  answer  these 

questions.  The  criteria  for  methodology  decisions  were  as 

follows:  (1)  the  data  gathered  should  be  as  complete  and 

accurate  as  possible,  (2)  the  analysis  should  use  proven 

statistical  methods  to  examine  the  data,  and  (3)  the 

inferences  drawn  should  be  accurate,  well  founded,  and  have 

a  high  degree  of  confidence. 

This  chapter  first  presents  the  classification  of 
research  and  reviews  the  experimental  design.  It  then 
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discusses  the  research  data,  including  the  data  source, 
measurement  techniques  associated  with  the  research,  the 
validity  of  the  research,  and  the  manipulation  of  the  data. 
Next,  it  briefly  considers  the  theory  and  application  of  the 
statistical  analysis  accomplished. 

Classification  of  Research 

This  was  a  formal  study  since  it  conducted  hypothesis 
testing  on  the  research  questions  using  analysis  of  variance 
and  pairwise  comparison  techniques.  It  was  observational  in 
nature  since  the  outcome  depended  on  available  data 
obtained  through  an  Air  Force  Systems  Command  database.  It 
was  also  ex  post  facto  since  there  was  no  control  over  the 
variables  in  the  study.  The  study  had  a  descriptive  purpose 
since  the  research  was  concerned  with  how  different 
categories  of  contracting  efforts  and  equipment/industrial 
classifications  affect  the  ability  of  an  Air  Force 
organization  to  meet  the  five  percent  SDB  goal.  The  study 
was  cross  sectional  in  nature  since  it  examined  only 
contracts  awarded  since  1987,  the  year  after  the 
Congressional  mandate  was  passed. 

Experimental  Design 

This  study  was  a  quasi-experiment  because  it  used 
observational  data  and  had  no  control  group  to  compare 
results  against.  The  experimental  design  was  a  two-factor 
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analysis  of  variance  as  depicted  in  Figure  2. 
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Figure  i  Statistical  Components 

The  first  factor  classified  a  contract  by  type  of 
contracting  effort.  The  Federal  Supply  Classification  (FSC) 
was  used  to  determine  four  factor  levels,  which  for  this 
study  included  (1)  Research,  Development,  Test,  and 
Evaluations  (RDT&E);  (2)  Services  and  Construction;  (3) 
Supplies  and  Equipment;  and  (4)  Uncoded.  Uncoded  entries 
were  either  difficult  to  classify  clearly  into  one  of  the 
categories  or  were  not  coded  due  to  error.  The  second 
factor  categorises  the  type  of  equipment  or  industry 
involved.  The  study  narrowed  nine  major  Supply  and 
Equipment  (SEC)  categories  to  five  because  some  of  the 
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categories  (i.e.  Aircraft  and  Aircraft  Engines,  Weapons  and 
Airanunition)  were  closely  related.  The  five  factor  levels 
were:  (1)  Aircraft  and  Aircraft  Engines,  (2)  Missiles  and 
Rockets,  (3)  Weapons  and  Ammvinition,  (4)  Electronics,  and 
(5)  Other  (which  included  vehicles,  ships,  and  \incoded 
entries).  The  data  points  which  had  no  SEC  code  were  also 
included  in  the  "Other"  category.  The  treatments  for  the 
study  were  the  combinations  of  the  two  factors  (i.e.. 
Electronics  and  Supplies  and  Equipment).  The  dependent 
variable  for  the  study  was  the  proportion  of  contract 
dollars  awarded  to  SDBs  for  each  treatment. 

Data 

Data  Source.  The  data  was  obtained  from  the  AFSC 
Acquisition  Management  Information  System  (AMIS)  database, 
located  at  Wright  Patterson  APB,  OH.  This  computer  database 
contains  all  the  information  from  DD  Forms  350  relevant  to 
AFSC.  The  major  sorting  factor  was  the  Federal  Supply 
Classification  (FSC)  code  and  the  minor  sorting  factor  was 
the  Supply  and  Equipment  Classification  (SEC)  code.  See 
Figure  3  for  a  representation  of  the  data  from  the  data 
base.  Notice  in  the  figure  the  FSC  codes  are  broken  down  in 
terms  of  four  classifiers  (the  four  digits).  The  SEC  codes 
are  broken  down  in  terms  of  three  classifiers.  These  more 
detailed  classifications  served  as  the  observational 
elements  which  went  into  the  treatments  of  the  study.  The 


37 


treatments  consisted  of  the  first  digit  or  classifier,  and 
in  some  cases,  a  combination  of  these  first  digits.  The 


"Total 
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were  not  included  as  datapoints. 
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Figure  3  Typical  Data  From  MilS  Database 


There  were  some  anomalies  in  the  data.  There  were  blank 
SEC  codes  and  some  SEC  codes  were  listed  as  "000".  It  is 
unknown  whether  the  coding  was  deficient  or  the  nature  of 
the  contract  made  no  code  appropriate.  These  anomalies  were 
common  in  the  data  and  resulted  in  an  overwhelming  amount  of 
the  data  (69  percent)  falling  into  the  "Other" 
classification  in  the  data  analysis.  If  these  datapoints 
would  have  been  in  the  titled  classifications  they  may  have 
added  to  the  knowledge  about  those  classifications.  There 
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were  also  some  datapoints  which  had  missing  PSC  codes  but 
these  were  few  (eleven  total). 

Even  though  these  anomalies  are  present  and  there  are 
potential  sources  of  error,  the  error  in  the  study  should  be 
minimal  since  the  individuals  dealing  with  it  were 
professionals  and  there  were  many  checks  against  error  in 
the  system. 

Measurement .  The  data  scale  type  involved  in  this  study 
is  ratio  since  it  consists  of  the  dollars  which  went  to  SDBs 
divided  by  the  total  contracting  dollars  for  each  specific 
treatment.  The  data  range  from  -.9351  to  2.235.  In  the 
cases  where  the  percentages  of  dollars  was  greater  than  100 
or  less  than  0,  the  SDB  dollar  amount  was  most  likely  either 
greater  than  the  total  dollar  amount  in  that  contracting 
category  because  some  of  the  non-SDB  contracts  had  money 
removed  from  a  contract  (a  negative  obligation)  or  some  of 
the  SDB  contracts  had  money  removed. 

Since  the  data  for  this  study  already  existed  and  is  not 
being  gathered  under  experimental  conditions,  it  is 
secondary  data.  The  advantages  of  secondary  data  is  that  it 
is  quick  to  access  and  collect  and  it  is  relatively  cheap. 
However,  this  is  at  the  expense  of  accuracy  and  consistency 
With  secondary  data  one  has  to  assume  the  accuracy  of  it 
since  it  was  not  gathered  and  assembled  under  carefully 
controlled  conditions.  Another  possible  drawback  with 
secondary  data  is  controlling  error. 
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Error .  In  terms  of  data  error  two  basic  sources  were  of 
concern:  human  and  instrumental.  In  this  study  human  error 
could  have  been  introduced  in  a  variety  of  ways.  The 
initial  coding  of  the  information  onto  the  form  at  the 
initiation  of  the  contract  could  be  in  error  (i.e.,  wrong 
code,  dollar  value,  type  of  business,  or  other  pertinent 
information).  When  the  form  reaches  the  person  responsible 
for  entering  the  data  into  the  database,  that  person  could 
make  the  same  mistake.  Or,  when  the  analysis  is  conducted 
on  the  information  retrieved  from  the  database,  the 
researcher  could  make  the  same  errors. 

Government  quality  control  procedures  should  minimize 
these  errors.  The  database  was  assumed  to  be  correct 
because  the  researcher  has  no  method  to  verify  or  test  its 
accuracy.  In  terms  of  the  researcher's  inputs,  the  data  was 
checked  and  cross-checked  for  accuracy. 

The  other  type  of  error,  instrument,  could  be  caused 
through  programming  errors  in  the  information  retrieval  or 
through  format  inconsistencies  with  the  requirements  of  the 
study.  Also,  if  the  statistical  package  analyzing  the  data 
were  to  err  during  the  analysis  (i.e.,  inappropriately 
applied  statistics  due  to  violated  statistics),  it  would 
introduce  instrument  error  into  the  analysis.  As  best  as 
the  researcher  could  determine  the  formats  were  all 
perfectly  compatible  and  the  statistical  packages  correct. 
Care  was  taken  to  use  appropriate  techniques,  however. 
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conditions  were  violated  which  degraded  thA  validity  of  the 
statistical  inferences.  This  will  be  discussed  in  more 
detail  in  the  Statistical  Analysis  section. 

Validity.  There  are  two  types  of  validity  for  data  and 
analysis:  external  validity,  or  the  interaction  of  the 
treatment  with  other  factors  and  the  resulting  impact  on 
abilities  to  generalize  across  time  setting  or  groups,  and 
internal  validity,  or  the  truthfulness  of  conclusions  drawn 
(9:115).  They  both  have  unique  characteristics  and  can 
threaten  the  validity  of  the  study  if  not  accounted  for. 

External  validity  can  be  threatened  in  two  ways.  The 
first  threat  to  validity  is  interaction.  If  the  data 
collected  is  interactive,  then  one  of  the  treatments  has  an 
effect  on  the  other.  If,  during  the  analysis,  assumptions 
are  made  and  conclusions  are  drawn  about  the  main  factors, 
the  conclusions  may  be  invalid  if  one  has  not  accounted  for 
interaction.  This  posed  a  problem  in  this  study  and  is 
addressed  in  detail  in  the  in  the  Statistical  Analysis 
section . 

The  second  threat  to  external  validity  is  data 
selectivity.  If  the  data  is  selective  in  nature;  e.g., 
choosing  specific  groups  to  analyze  in  stead  of  the  total 
database,  then  the  results  will  also  be  invalid  since  they 
cannot  be  generalized  across  time,  settings  or  groups.  This 
study  used  selective  data  (AFSC  data  for  1987-1989)  and 
therefore  may  not  be  generalizable  outside  of  the  AFSC  or 
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the  years  included  in  the  study  (1987-1989). 

In  terms  of  internal  validity,  the  points  mentioned 
earlier  regarding  error  in  the  data  are  applicable.  Any 
sources  of  error  in  the  data  will  affect  the  internal 
validity.  In  addition  to  this,  there  are  other  items 
affecting  internal  validity  which  must  be  addressed. 

Any  changes  in  data  collection  can  affect  internal 
validity.  For  example,  the  DD  Forms  350  changed  format 
during  the  period  of  time  under  study.  However,  the 
portions  affecting  this  study  did  not  change.  No  other 
changes  occurred  which  could  impact  the  internal  validity  of 
this  study. 

The  computer  and  statistical  analysis  can  affect 
internal  validity,  too.  If  the  computer  or  the  statistical 
analysis  are  not  as  accurate  as  the  study  dictates,  error 
will  be  introduced  and  validity  will  be  threatened.  Since 
this  study  will  require  a  relatively  simple  Analysis  of 
Variance  (ANOVA)  and  generally  accepted  statistical  software 
was  used,  this  was  not  a  problem. 

Accuracy  of  the  data  can  be  a  threat  to  internal 
validity,  also.  If  the  data  in  the  database  does  not 
accurately  reflect  the  actual  real  life  data,  the  results 
can  be  invalid.  The  accuracy  of  the  data  has  already  been 
addressed. 

Data  Manipulation.  As  a  result  of  the  data  format,  the 
first  effort  of  the  study  was  to  gather  the  more  detailed 
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four  digit  FSC  and  SEC  codes  into  the  aggregated  single 
digit  categories.  This  was  accomplished  by  hand  conversion. 
There  were  2849  data  points  altogether.  Due  to  its  volvune, 
the  actual  data  will  not  be  included  in  this  report. 

The  raw  data  was  loaded  into  QUATTRO  (software  by 
Borland)  by  hand.  It  was  then  transferred  to  STATISTIX 
(software  by  NH  Analytical  Software)  and  used  to  gather  some 
descriptive  statistics  on  the  data.  They  include  the  mean, 
standard  deviation,  number  of  data  points,  the  median,  and 
the  minimum  and  maximum  values  for  each  treatment.  The  raw 
data  was  then  uploaded  to  the  AFIT  VAX  computer  so  the  ANOVA 
tests  could  be  accomplished  using  the  SAS  statistical 
program.  The  raw  data  was  then  organized  into  a  QUATTRO 
spreadsheet  in  order  to  accomplish  a  Tukey  pairwise 
comparison  of  means. 

Statistical  Analysis 

This  study  used  descriptive  statistics  to  answer  the 
first  research  question  and  conducted  two-factor  tests  to 
answer  the  second  research  question.  First,  the  aptness  of 
the  data  for  the  models  used  was  analyzed.  Descriptive 
statistics  and  95  percent  confidence  intervals  were  used  to 
determine  whether  the  means  of  each  treatment  were  equal  to 
.05,  the  SDB  goal.  To  accomplish  the  two-factor  testing, 
the  study  used  ANOVA  testing  methods  and  pairwise  comparison 
of  means  procedures.  The  remainder  of  this  chapter 
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discusses  the  theory  behind  this  analysis. 

Aptness  Analysis.  Before  performing  any  statistical 
tests,  the  study  first  analyzed  the  aptness  ot  the  data  to 
be  used  in  the  parametric  models  which  performed  the 
statistical  analysis.  If  the  data  is  not  (•pt  for  the  models 
used,  the  results  of  the  analysis  will  be  unsound  and 
suspect.  The  aptness  analysis  was  accomplished  by 
attempting  to  validate  the  assumptions  of  the  data.  These 
assumptions  are: 

1.  The  treatment  data  are  normally  distributed. 

2.  The  treatment  data  have  equal  variances. 

3.  The  treatment  data  are  random  and  independent. 
Since  the  data  is  a  census,  the  third  assumption  is  not 
necessary.  However,  the  first  two  assumptions  need  to  be 
tested . 

This  was  done  by  looking  at  the  residuals  of  the 
treatment  data  and  performing  Wilk-Shapiro  tests  on  the 
data.  The  residual  plots  are  defined  as: 

®ij  =  ^ij  ■  Yhatij  =  Yij  -  Ybari. 

The  residual  plots  were  determined  using  SAS.  By  testing 
for  (1)  nonconstancy  of  error  variance,  (2)  outlier  data, 

(3)  omission  of  important  independent  variables,  and  (4) 
nonnormality  of  error  terms  one  can  check  for  the  accuracy 
of  these  assumptions. 
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In  order  to  check  for  nonconstancy  of  error,  the 
residual  plots  should  show  about  the  same  extent  of  scatter 
of  the  residuals  around  0  for  each  factor  level.  The 
residuals  for  the  data  from  this  study  did  not  show  the  same 
extent  of  scatter.  Therefore,  nonconstancy  of  error  is  an 
invalid  assumption.  For  outliers,  one  needs  to  see  if  there 
are  outlier  observations  and  then  accept,  correct,  or  reject 
the  data  point  if  deemed  necessary.  Based  on  the  plots, 
there  are  a  couple  of  outlier  data  points.  These  were  the 
same  points  that  were  greater  than  1  and  less  than  0  during 
the  descriptive  statistics.  These  points  were  kept  in  the 
an. lysis  because  they  could  not  be  considered  incoirect 
data.  If  there  appears  to  be  some  factor  which  when 
examined  leads  the  analyzer  to  believe  he/she  may  separate 
the  residual  plots  into  distinctive  groups,  a  more  detailed 
multifactor  analysis  may  be  in  order.  This  does  not 
invalidate  the  original  model,  however,  it  shows  more 
analysis  should  be  accomplished.  The  residual  plots  did  not 
indicate  any  patterns  suggesting  a  variable  should  be  added. 
To  check  nonnormality  of  the  error  terms,  a  histogram  of  the 
residuals  was  checked  to  see  if  they  followed  a  normal 
distribution.  The  histograms  indicated  the  data  was 
extremely  nonnormal. 

In  addition  Wilk-Shapiro  tests  with  rankit  plots  were 
conducted  to  determine  normalness  of  the  data  using 
STATISTIX.  The  results  are  shown  in  Appendix  A.  If  the 
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data  was  normally  distributed,  the  data  points  on  tho  rankit 
plot  would  be  linear  and  about  at  a  forty  five  degree  slope 
from  the  origin  and  the  Wilk'Shapiro  test  should  result  in  a 
number  between  .9  and  1.  Based  on  the  results  of  these 
tests,  none  of  the  data  sets  could  be  assumed  normal  or  even 
approximated  as  normal . 

These  results  indicate  the  data  did  not  satisfy  the 
conditions  for  using  parametric  models  and  perhaps  some 
nonparametric  tests  which  can  overcome  the  problems  of 
nonnormal  data  should  have  been  used. 

The  nonparametric  tests  generally  use  a  ranking 
procedure  to  alter  the  data  so  it  is  usable  by  some 
statistical  model.  However,  one  drawback  of  this  procedure 
is  that  if  the  analysis  shows  the  treatments  result  in 
different  means  then  the  cause  of  the  difference  cannot  be 
determined.  Some  authors  have  addressed  this  shortcoming. 
According  to  Jerome  Myers,  in  his  book  Fundamentals  of 
Experimental  Design,  nonparametric  tests  were  not 
appropriate.  He  states: 

The  use  of  nonparametric  statistics  .  .  .  are  also 
sensitive  to  differences  in  other  parameters  besides  the 
mean  and  are  not  necessarily  more  powerful  than  the  F 
test  when  the  assumptions  of  that  test  are  violated. 

The  main  reason  for  my  restraint  in  using  nonparametric 
statistics  is  simply  that  they  are  not  versatile  enough, 
that  researchers  who  use  the  nonparametric  approach  are 
limited  in  the  designs  they  can  use  and  what  they  can 
ask  of  their  data  (19:75). 

The  choice  the  study  was  left  with  was  to  use  the  original 
models  with  the  understanding  that  the  normality  assumption 
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has  not  been  met.  One  must  understand  the  results  may  or 
may  not  be  valid  based  on  this  inability  to  meet  the 
assumption  of  normality.  However,  until  a  statistical 
process  is  found  that  will  accomplish  the  objectives  of  the 
study,  the  models  used  are  the  best  statistical  models 
available. 

Research  Question  t  1.  Using  STATISTIX,  confidence 
intervals  were  generated  for  each  treatment  to  determine 
whether  they  did  not  meet,  met,  or  exceeded  the  .05  level 
goal.  The  test  used  an  a  of  .05  (95  percent  confidence 
level).  The  confidence  interval  analysis  is  basically  a 
hypothesis  test  with  the  following  hypotheses: 

Ho:  The  treatment  mean  equals  .05. 

Hj:  the  treatment  mean  is  not  equal  to  .05. 

If  the  confidence  interval  contains  .05,  the  mean  equals  .05 
with  a  95  percent  level  confidence.  The  confidence 
intervals  are  based  on  the  treatment  means  and  standard 
deviations  combined  with  the  appropriate  T  value  (a  two- 
tailed  T  test).  The  results  of  this  test  are  discussed  in 
the  nexw  chrpter. 

Research  Question  f  2.  The  second  research  question  was 
answered  using  a  two-factor  analysis.  There  are  several 
advantages  of  performing  two-factor  analyses  instead  of 
single  factor  analyses.  They  are  more  efficient  studies 
since  two  factors  can  be  compared  in  the  same  test,  more 
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information  is  gathered  during  the  test,  and  the  validity  of 
the  study  can  be  increased  by  considering  interaction 
between  the  two  factors.  The  additional  information  regards 
interaction  between  the  treatments  and  factor  levels  and 
cross  treatment  of  the  same. 

This  analysis  consisted  of  three  parts:  (1)  to  perform  a 
factor  effects  analysis  to  determine  if  there  is 
interaction;  (2)  to  transform  the  data,  if  necessary  and  if 
possible,  to  negate  the  interaction  effects;  and  (3)  to 
answer  the  second  research  question  using  a  pairwise 
comparison  of  means.  The  strategy  for  conducting  this 
analysis  is  depicted  in  Figure  4. 

Factor  Effects  Analysis.  The  first  step  of  two- 
factor  analysis  was  to  determine  if  interaction  between  the 
two  factors  was  present.  Interaction  is  important  because 
it  masks  the  main  effects  of  the  two  factors.  If  no 
interaction  is  present,  the  factors  can  be  described 
separately  by  analyzing  the  main  effects  (20:672).  There 
are  many  ways  to  determine  whether  or  not  there  is 
interaction: 

1.  If  =  p.  .  +  +  flj  there  is  no  interaction. 

2.  If  the  difference  between  the  mean  responses  for  any 

two  levels  of  factor  B  is  the  same  for  all  levels  of 

factor  A  there  is  no  interaction. 

3.  If  the  difference  between  the  mean  responses  for  any 

two  levels  of  factor  A  is  the  same  for  all  levels  of 

factor  B  there  is  no  intera<^t:ion. 
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4.  If  the  curves  of  the  mean  responses  for  the 
different  levels  of  a  factor  are  all  parallel  there  is 
no  interaction. 

If  there  is  interaction,  there  are  two  different 
classifications.  Important  interactions  imply  one  cannot 
discuss  or  conclude  the  effects  of  each  factor  separately  in 
terms  of  the  factor  level  means.  In  contrast,  unimportant 
interaction  implies  the  interaction  effects  are  small  and 
the  analysis  can  be  the  same  as  with  no  interaction  at  all 
(20:679).  This  distinction  is  subjective,  however,  and  is 
left  to  the  discretion  of  the  analyzer. 

To  test  for  interaction,  the  study  used  a  fixed-effect 
model  because  the  factor  levels  were  chosen  for  their 
intrinsic  interest  and  they  were  not  considered  a  sample 
from  a  larger  population  (20:523).  One  complication  arose 
because  the  treatments  had  different  sample  sizes.  The 
unequal  sample  sizes  cause  (1)  the  least  squares  equations 
normally  used  for  ANOVA  to  be  complicated  and  (2)  the  factor 
effect  component  sum  of  squares  to  no  longer  be  orthogonal 
which  leads  to  an  inability  to  sum  to  8STR  (Svun  of  Squares 
of  TReatments).  As  a  result,  a  general  ANOVA  was  performed 
with  the  General  Linear  Model  (GLM)  which  allowed  for 
correct  analysis  even  with  unequal  sample  sizes. 

The  OLM  has  several  elements: 

A.  Treatment  Mean 
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B.  Factor  Level  Means 


U- 


j 


Wi-  = 
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2:uij 

i=l 


a 

b 
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where  a  =  #  of  j  treatments 


where  b  =  #  of  j  treatments 


C.  Overall  Mean  = 


i  j 

p.  .  = - 

ab 

D.  Specific  Effects 
®j(i)  “  ^ij  “ 


E.  Main  Effects 


^Hi) 

1 

=  — . =  p.j  -  W..  +  Zflj  =  0 

a  ] 

3 

ft.  - - -  =  0 

b  i 


The  OLM  now  needs  to  be  fitted  for  each  test  of  factor 
interactions  (20:747). 
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The  fitted  model  is  defined  as: 

Yijk  =  u..  +  tti  +  flj  +  (aB)ij  +  Cijk 

where:  y..  is  a  constant 

is  a  constant  subject  to  =  0 

Bj  is  a  constant  subject  to  £Bj  =  0 

(aB)^j  are  constants  subject  to 

2:(dtB)^j  =  0  and  z:(aB)£j  =  0 

i  j 

Ciji^  are  independent  and  N(0,o^) 
and  i  =  l...a,  j  =  l...b,  k  =  l...n 
In  this  study  a  equaled  4  and  b  equaled  5  since  this  is  how 
many  factor  levels  there  are  for  each.  The  degrees  of 
freedom  for  the  a  factor  are: 

a-l  =  4-  l*3 
and  for  the  b  factor  are: 

b-l*5“ls4. 

The  parameters  are: 

y.  . 

Oi  =  yi-  -  y.  . 

Bj  =  P.j  -  U*. 

(aB)ij  =  yjj  -  yj.  -  yj.  ♦  y.  . 

When  testing  for  interaction  effects  with  the  regression 
model  the  following  hypothesis  test  was  accomplished: 

Hq:  all  (aB)^j  =  0 

H,:  not  all  (aB)jj  =  0 
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In  order  to  accomplish  this,  an  F  test  was  done  where 


^  SSE(R)  -  SSE(F)  SSE(F) 

p . - . / . 

d£]{  ~  dfp  dfp 

If  a  is  set  at  .05,  P(.95,  dfp  -  dfp,  dfp)  =  X  and  if  F*  <  X, 
there  is  no  interaction. 

To  accomplish  this,  data  matrices  were  designed  in  order 
for  the  SAS  statistical  package  to  analyze  the  data.  It 
consisted  of  an  x  matrix  and  a  y  matrix.  Basically,  the  x 
matrix  was  designed  so  the  elements  of  the  data  set  (factor 
levels  and  treatments)  could  be  analyzed  using  matrix 
algebra.  The  actual  data  was  designed  as  a  y  matrix.  Due 
to  the  size  of  the  matrices  they  were  not  included  in  the 
study.  The  x  matrix  was  set  equal  to  the  y  matrix  so  the 
treatments  and  factor  levels  become  the  dependent  variables 
and  the  data  becomes  the  independent  variable. 

The  hypothesis  to  test  for  interaction  was: 

Hq:  All  the  interaction  terms  are  equal. 

H,:  At  least  one  treatment  mean  is  different. 


The  test  showed  interaction  is  present.  This  is 
discussed  in  more  detail  in  the  Findings  and  Analysis 
chapter . 

In  addition  to  this  GLM  test,  an  analysis  of  interaction 
was  accomplished  graphically.  The  lines  resulting  from 
plotting  the  factor  levels  of  interest  with  the  values  of 
the  alternate  factor  levels  serving  as  data  points  were  not 
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parallel,  indicating  interaction.  See  the  next  chapter  for 
the  graphed  interaction  plots  and  further  analysis. 

Transforming  Data.  The  next  step  in  the  analysis 
would  be  to  perform  some  type  of  transformation  on  the  data 
to  attempt  to  correct  the  above  shortcomings.  However, 
based  on  the  specific  nature  of  this  data,  transformation 
was  infeasible.  No  simple  form  of  transformation  could  be 
accomplished  which  will  allow  main  factors  to  be  analyzed. 

Since  there  was  interaction  present  and  it  could  not  be 
transformed,  the  study  could  not  determine  main  factor 
effects  (whether  there  is  a  significant  difference  in  means 
between  the  main  factors;  i.e.,  PSC  factor  levels  and  SEC 
factor  levels).  The  only  possibility  for  further  analysis 
was  to  test  for  simple  main  effects  using  a  pairwise 
comparison  of  means  within  each  factor  level. 

Pairwise  Comparison  of  Means.  The  pairwise 
comparison  of  means  was  accomplished  through  a  Tukey 
analysis.  The  Tukey  method  of  analysis  uses  the  studentized 
range  distribution  and  does  pairwise  combination  comparisons 
of  the  means  to  determine  if  there  is  a  significant 
difference  between  any  of  the  factor  means.  This  allows  the 
researcher  the  opportunity  to  see  which  of  the  treatments 
account  for  the  difference  in  means.  The  analysis  was 
accomplished  using  QUATTRO  and  the  formulas  for  a  Tukey 
pairwise  comparison  of  means  as  given  in  Neter,  Hasserman 
and  Kutner  (1985).  The  QUATTRO  output  is  given  in  Appendix 
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B.  The  objective  was  to  determine  those  treatments  which 
differed  significantly  from  the  others  based  on  their  means, 
their  MSE  (mean  standard  error)  and  their  standard 
deviations.  A  difference  between  the  treatments  of  interest 
is  determined  and  then  a  confidence  interval  was  calculated 
based  on  the  aforementioned  parameters.  If  the  resulting 
confidence  interval  contains  sero,  then  the  two  treatments 
are  not  significantly  different.  However,  if  they  do  not 
contain  zero,  then  the  treatments  do  differ  significantly. 
This  may  lead  to  some  conclusions  about  contracting 
categories  factors  which  make  it  easier  or  more  difficult  to 
meet  the  goal.  The  results  of  the  analysis  are  discussed  in 
the  next  chapter. 

Conclusion 

This  chapter  discussed  the  methodology  used  to  perform 
the  tests  to  answer  the  research  questions.  The  collected 
data  was  found  to  not  be  apt  for  the  parametric  models  used, 
however,  no  better  models  exist  which  can  perform  the  tests 
so  the  parametric  models  will  be  used.  However,  this  may 
cause  the  results  of  the  tests  to  be  suspect.  The 
statistical  models  were  used  to  accomplish  a  confidence 
interval  analysis  which  attempted  to  answer  the  first 
research  question  and  ANOVA  procedures  were  used  to  answer 
the  second  research  question.  The  next  chapter  discusses 
the  results  of  these  tests. 
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IV.  Findings  and  Analysis 


Introduction 

The  objective  of  this  research  was  to  determine  whether 
the  type  of  contracting  effort  and  equipment/industry 
classification  impacted  the  ability  of  AF  organizations  to 
meet  the  five  percent  Small  and  Disadvantaged  Business  (SDB) 
goal.  This  chapter  focuses  on  the  results  of  the  data 
collection  and  statistical  analysis  as  they  pertain  to  the 
following  research  questions: 

1.  Does  the  percentage  of  contract  dollars  going  to 
SDBs  meet  the  five  percent  goal ,  given  the  type  of 
contracting  effort  and  the  specific  industry  or  product 
classification? 

2.  Does  the  percentage  of  contract  dollars  going  to 
SDBs  significantly  vary  between  category  of  contracting 
effort  and  equipment/industry  classification? 

This  chapter  presents  the  findings  of  the  research  based  on 

Air  Force  Systems  Command  (AFSC)  contracts  covering  the 

years  1987-1989.  Specifically,  it  contains  some  general 

findings,  the  findings  of  the  confidence  interval  generation 

conducted  to  answer  the  first  research  question,  and  the 

Analysis  of  Variance  (ANOVA)  analysis  which  includes  a 

pairwise  comparison  of  means  to  answer  the  second  research 

question. 
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General  Findings 


A  summary  of  the  data  is  presented  in  Figure  5.  It 
provides  the  mean  proportion  of  dollars  awarded  to  SDB 
firms,  the  standard  deviation,  and  the  number  of  sub¬ 
classifications  which  served  as  observational  elements  for 
each  combination  of  contracting  effort  and 
equipment/industry  classification.  The  •*='',  and  ">" 

symbols  indicate  whether  the  mean  proportions  are 
statistically  equal  to,  less  than,  or  greater  than  the  five 
percent  SDB  goal  established  by  Congress.  The  following 
abbreviations  are  used  in  the  figure  and  throughout  the 
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Figure  S  Sunmarized  Data  Matrix 
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remainder  of  the  study: 


Air 

Aircraft 

Eng 

Aircraft  Engines 

Msl 

Missiles 

Rkt 

Rockets 

Hpn 

Weapons 

Ammo 

Ammunition 

Elec 

Electronics 

RDT&E 

Research,  Development,  Test, 
and  Evaluation 

Sup 

Supplies 

Equip 

Equipment 

Svcs 

Services 

Const 

Construction 

The  first  general  observation  is  the  wide  range  of  means  and 
standard  deviations.  Several  cells,  most  notably  the  ones 
associated  with  Weapons  and  Ammvmition  contracts  as  well  as 
Uncoded  Federal  Supply  Classifications,  have  means  of  zero 
and  small  standard  deviations.  Such  statistics  suggest 
there  are  pockets  of  contracting  efforts  which  are  clearly 
having  difficulties  meeting  the  five  percent  goal. 

Similarly,  there  are  some  cells  with  quite  large  means, 
ranging  from  nearly  10  percent  to  over  30  percent  of 
contract  dollars  awarded  to  small  and  disadvantaged  firms. 
However,  these  are  marked  by  quite  large  standard 
deviations,  suggesting  the  data  is  very  dispersed.  In  part 
this  was  due  to  the  fact  that  many  sub-elements  would  tend 
to  be  all  or  nothing  in  terms  of  SDB  awards.  These  areas 
apparantly  have  pockets  of  great  opportunity  for  conducting 
business  with  SDB  firms.  Thus,  the  data  suggest  that 
generally  the  opportunity  or  ability  to  contract  with  SDBs 


58 


is  not  consistent  across  categories  of  contracting  efforts 
or  types  of  industry  and  equipment  involved.  More  specific 
trends  will  be  discussed  later.  The  wide  range  of  variance 
also  indicates  that  the  assumption  of  homogeneity  of 
variance  required  by  parametric  statistical  techniques  has 
been  violated.  This  was  addressed  in  some  detail  in  the 
methodology  chapter.  This  limits  the  validity  and  certainty 
with  which  statistical  inferences  can  be  made.  This  should 
be  kept  in  mind  when  considering  the  results  of  the 
statistical  tests. 

The  large  variation  in  observations  for  the  cells  are 
due  to  the  number  of  sub-categories  falling  into  each  pair 
of  categories  (type  of  contracting  category  and  type  of 
equipment/ industry  involved).  Of  special  interest  is  the 
large  number  falling  into  the  "Other"  category.  In  fact, 
over  68  percent  of  the  total  datapoints  were  in  this  area. 
This  was  a  result  of  either  not  coding  in  the  SEC 
classification  or  coding  in  "000".  This  classification  also 
included  vehicles  and  ships,  but  there  were  very  few 
datapoints  from  these  classifications  (three).  The  means 
for  this  area  were  generally  quite  high,  suggesting  fairly 
good  opportunity  for  SDB  contracting.  Unfortunately,  the 
lack  of  visibility  in  this  area  does  not  permit  detailed 
analysis . 

There  were  some  FSC  datapoints  which  were  uncoded.  The 
unfortunate  result  of  these  uncoded  FSC  and  SEC  entries  and 
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miscoded  SEC  entries  is  that  instead  of  this  data  falling 
into  its  appropriate  category,  it  fell  into  a  catch-all 
category.  With  no  background  information  on  the  specific 
contracts,  no  conclusions  can  be  reached  on  the  data  from 
these  uncoded  categories,  other  than  there  appears  to  be 
varying  levels  of  SOB  contracting. 

One  problem  encountered  during  the  statistical  analysis 
was  that  the  assumption  that  the  treatment  data  sets  had 
normal  distributions  vas  not  met.  This  assumption  was 
investigated  through  a  residual  analysis  and  Hilk-Shapiro 
rankit  plots.  The  result  of  this  investigation  showed  the 
data  sets  were  far  from  normal.  This  would  normally  force 
nonparametric  test  methods.  However,  it  was  determined  the 
originally  intended  model  would  work  as  well  as  a 
nonparametric  test  because  of  the  ratio  data  involved.  The 
drawback  is  that  the  results  of  the  analysis  may  be  suspect 
because  of  this  invalidated  assumption.  Until  a  better 
model  is  developed,  this  is  the  best  analysis  that  can  be 
done . 

Research  Question  t  1 

The  first  research  question  addressed  whether  or  not  the 
ability  to  meet  the  five  percent  goal  was  dependent  on  the 
type  of  contracting  effort  and  the  industry/equipment 
category  involved.  In  order  to  answer  this,  95  percent 
confidence  intervals  were  established  for  each  cell  of 
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Figure  5.  This  confidence  interval  suggests  that  if  the 
data  were  normally  distributed,  and  repeated  random 
samplings  were  taken,  the  true  proportion  of  contracts 
awarded  to  SDB  firms  would  be  captured  95  percent  of  the 
time.  In  this  case,  if  the  confidence  interval  included 
.05,  the  cell  was  judged  not  to  be  statistically  different 
and  thus  equal  to  the  fiver  percent  goal.  These  cells  are 
annotated  with  an  symbol.  If  the  confidence  interval 
was  lower  than  .05,  it  was  judged  to  be  lower  than  the  goal 
(indicated  by  "<"  in  the  figure).  Similarly,  intervals 
above  .05  were  designated  as  greater  than  the  five  percent 
goal  (indicated  by  ">*'). 

The  treatments  which  did  not  meet  the  goal  were 
clustered  mainly  in  the  Weapon  and  Ammunition 
equipment/industry  classification  and  the  Uncoded 
contracting  category.  Given  the  means  and  standard 
deviations  of  zero,  it  appears  that  contracting  in  the 
Weapon  and  Anmunition  classifications  has  a  difficult  time 
meeting  the  five  percent  goal,  at  least  in  the  APSC 
environment,  which  could  be  contracting  for  research  and 
development,  production,  and  deployment  of  new  systems. 

That  may  or  may  not  be  true  when  contracting  for  spares  and 
other  support.  It  is  difficult  to  interpret  the  lack  of  SDB 
contracting  for  the  uncoded  FSCs. 

Electronic  supplies  and  e^  ' pment  also  had  a  mean  and 
standard  deviation  of  zero.  Thus,  systems  contracting  in 
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this  area  appears  to  be  restricted.  It  may  be  that  there 
are  very  limited  numbers  of  SDBs  that  have  the  technical 
capability  to  make  high  technology  electronic  components, 
even  though  there  appear  to  be  SDB  firms  capable  of 
performing  RDT6E  efforts.  It  is  possible  that  there  might 
be  SDBs  capable  of  producing  replenishment  spares.  Most  of 
the  Supply  and  Equipment  contracting  for  spares  performed  in 
the  AF  is  performed  by  the  Air  Force  Logistics  Command 
(AFLC).  For  this  reason,  data  obtained  from  the  AFLC  may 
indicate  a  greater  opportunity  for  contracts  in  this  area 
and  higher  number  of  SDB  business. 

The  treatments  which  met  the  goal  were  concentrated  in 
the  RDTSE  and  the  Supply  and  Equipment  contracting 
categories.  They  contained  five  of  the  six  treatments  which 
met  the  goal.  The  Aircraft  and  Aircraft  Engine,  Missile  and 
Rocket,  and  Electronic  classifications  in  the  RDTSE  category 
had  a  substantial  number  of  data  points  and  met  the  goal . 
This  indicates  the  RDT&E  and  the  Supply  and  Equipment 
categories  not  only  have  opportunities  to  use  SDB 
businesses,  but  have  a  sufficient  emphasis  and  number  of  SDB 
contractors  to  meet  the  goal.  The  "Other”  classification 
for  the  RDTSE  efforts  supports  this  conjecture  since  it 
shows  the  SDB  percentage  of  contracting  dollars  exceeds  the 
goal  . 

The  Supply  and  Equipment  category  met  the  goal  in  the 
Aircraft  and  Aircraft  rUngine  classification  and  the  Missile 
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and  Rocket  classification.  This  indicates  there  is  ample 
opportunity  for  contracts  in  these  areas  and  there  is  enough 
emphasis  and  are  enough  SDB  contractors  to  meet  the  goal . 

The  "Other"  classification  within  this  category  supports 
this  conjecture  since  it  exceeds  the  goal . 

The  classifications  which  had  a  notable  number  of 
treatments  which  met  the  goal  were  the  Aircraft  and  Aircraft 
Engine  classification  and  the  Missile  and  Rocket 
classification.  Except  for  the  Uncoded  categories,  all  the 
treatments  met  or  exceeded  the  goal .  The  categories  in 
these  classifications  which  had  a  substantial  number  of 
datapoints  (RDT&E  and  Supply  and  Equipment)  met  the  goal  but 
the  category  which  had  significantly  fewer  datapoints 
(Services  and  Construction)  exceeded  the  goal.  This 
indicates  that  within  the  RDT&E  and  the  Supply  and  Equipment 
contracting  efforts  there  was  a  wide  range  of  sub-* 
classifications  available  for  SDB  contracting  yet  they  could 
only  meet  the  goal  whereas  in  the  Services  and  Construction 
effort  there  were  fewer  sub-classifications  to  meet  the  goal 
in  yet  the  data  exceeded  it.  It  appears  that  systems 
contracting  efforts  for  the  Aircraft  and  Aircraft  Engine 
products  and  in  the  Missile  and  Rocket  industries  have 
considerable  services  and  construction  contracting 
opportunities  and  that  a  relatively  high  proportion  of  these 
can  go  to  SDBs . 

The  treatments  which  exceeded  the  goal  were  clustered  in 
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the  Services  and  Construction  contracting  category  and  the 
"Other"  industry  or  product  classification.  All  of  the 
"Other"  treatments  except  the  treatment  from  the  Uncoded 
category  and  all  the  Service  and  Construction  treatments 
except  the  treatment  from  the  Weapons  and  Ammunition 
classification  exceeded  the  goal.  This  indicates  there  is 
sufficient  opportunity  in  the  Services  and  Construction 
contracting  efforts  to  hire  SDBs  and  ample  SDBs  to  perform 
the  contracts.  The  "Other"  category's  ability  to  exceed  the 
goal  indicates  that  there  is  enough  interest  in  SDBs  and 
enough  SDBs  performing  to  allow  these  "hard  to  code"  or 
uncoded  industry  efforts  to  exceed  the  goals.  These  areas, 
if  more  visible,  might  offer  some  suggestions  to  increase 
opportunities  of  SDB  contracting. 

This  section  showed  that  some  treatments  were  below  the 
goal ,  some  met  the  goal ,  and  some  exceeded  the  goal . 
Moreover,  it  suggests  a  pattern  of  opportunity  may  exist, 
the  second  research  question  examines  whether  any  general 
patterns  exist  in  terms  of  classification  of  contracting 
efforts  and  the  type  of  equipment/industry  involved. 

Research  Question  t  2 

To  answer  the  second  research  question  an  ANOVA  was 
performed  on  the  data.  The  original  goal  was  to  determine 
whether  or  not  there  is  a  significant  difference  in  means 
between  the  main  factors  (i.e.,  FSC  factor  levels  and  SEC 
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factor  levels).  However,  complications  arose  during  the 
process.  The  ANOVA  process  has  three  basic  steps,  (1) 
determine  if  there  is  interaction  between  the  factors  of 
interest,  (2)  if  there  is  interaction,  transform  the  data  so 
the  interaction  is  not  a  factor,  and  (3)  perform  the 
analysis  to  answer  the  research  question. 

The  first  step  was  accomplished  using  the  General  Linear 
Model  (GLM)  to  perform  an  F  test.  The  objective  of  the  F 
test  procedure  is  to  answer  the  following  hypothesis: 

Hq:  All  the  interaction  terms  are  equal  to  0. 

At  least  one  interaction  term  is  not  equal  to  0. 

This  was  accomplished  using  SAS  statistical  software.  The 
results  are  shown  in  Figure  6. 

Dependent  Variable:  Y 

Te.^t:  Numerator:  0.2010  DF:  11  P  value:  2.2721 

Denominator:  0.088482  DF:  2828  Pr:b>F:  0.0094 

Fijure  6  SAS  Hypothesis  Testing  Results 

Slice  the  p  value  (Prob>F  or  .0094)  is  less  than  the  o  value 
fc  r  the  desired  confidence  level  (a  is  equal  to  .05  for  the 
desired  confidence  level  of  95  percent),  the  test  shows 
th^re  is  interaction. 

Another  method  used  to  determine  whether  interaction  is 
present  is  to  plot  the  factor  levels  of  interest  with  the 
values  of  the  alternate  factor  levels  serving  as  datapoints 
in  that  factor  level.  If  there  is  no  interaction  the  lines 
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should  be  parallel.  The  results  of  this  method  are 
presented  in  Figures  7  and  8. 


Figure  7 Test  for  Interaction 


The  two  figures  not  only  showed  interaction,  but 
significant  interaction.  However,  in  addition  to  showing 
interaction,  the  figures  highlight  some  general  patterns. 

In  Figure  7  the  percentage  of  SDB  efforts  in  the  Services 
and  Construction  category  of  contracting  is  higher  than  all 
the  other  areas  except  for  within  the  Weapons  and  Ammunition 
industry  classification.  In  Figure  8,  the  lines  all  follow 
the  same  trend  only  with  different  magnitudes.  Within  the 
RDT&E  category,  all  the  equipment/industry  classifications 
are  clustered  around  .05.  All  the  classifications  then 
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ROT&E  SVCSACONST  SUP&EQUP  UNCOOED 

_  _ FSC  CODES _ 

Figure  8  Test  For  Interaction 

increase  for  the  Services  and  Construction  area,  and 
decrease  for  the  Supply  and  Equipment  category.  The 
classifications  then  decrease  again  for  the  Uncoded 
category.  In  general,  the  Services  and  Construction 
category  has  a  higher  percentage  than  the  other 
classifications.  These  results  correlate  with  those 
discussed  in  the  previous  section  on  confidence  intervals. 

The  previous  figures  showed  significant  interaction.  In 
fact,  the  interaction  is  so  significant  that  no 
transformation  will  allow  the  main  factors  to  be  compared. 
Therefore  the  analysis  of  main  effects  will  not  be 
accomplished.  The  analysis  to  determine  whether  there  is  a 
significant  difference  between  the  means  will  be  confined  to 


analyzing  the  treatments  within  a  factor  level.  This  allows 
the  analyzer  to  focus  on  any  one  particular  level  rather 
than  a  general  analysis  across  all  treatments  and  factor 
levels.  The  next  step  is  to  perform  a  pairwise  comparison 
of  means  to  determine  whether  there  are  significant 
differences  in  the  treatment  means  within  a  given  category 
or  classification  and  which  are  significantly  different. 

This  was  accomplished  with  a  Tukey  pairwise  comparison  of 
means . 

The  Tukey  analysis  generates  confidence  intervals  for 
each  pair  of  treatments  under  investigation  and  then 
determines  whether  the  treatment  means  can  be  considered  the 
same  within  a  specified  confidence  level  (95  percent  in  this 
study).  It  is  a  relatively  conservative  approach  since  it 
uses  the  studentized  T  distribution  in  its  analysis.  The 
Tukey  analysis  in  this  study  was  performed  with  QUATTRO 
software  and  the  formula  for  Tukey  generation  for  variables 
with  unequal  sample  sizes  from  Neter,  Wasserman  ,and  Kutner 
(1985)  . 

The  standard  way  to  show  the  results  of  a  Tukey  analysis 
is  to  underline  those  treatments  which  do  not  differ 
significantly.  In  some  cases  one  or  more  treatments  will  be 
underlined  more  than  once.  In  this  case,  the  ones 
underlined  with  the  same  line  are  not  significantly 
different  but  those  treatments  not  underlined  with  the  same 
line  are  significantly  different.  A  summary  of  the  output 
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for  the  SEC  code  as  the  major  factor  follows. 

SEC  Codes  as  Maior  Factors.  The  results  of  the  Tukey 
Analysis  with  the  equipment/industry  classifications  as  the 
major  factors  are  shown  in  Figure  9. 

The  Services  and  Construction  contracting  category  is 
significantly  different  from  the  RDT&E  and  Supplies  and 
Equipment  contracting  categories  in  all  the 

Aircraft  and  Engine  Code 

SVCS&CONST  RDT&E  SOP&EQUIP  UNCODED 


Missile  and  Rocket  Code 

SVCS&CONST  UNCODED  RDT&E  SUP&EQUIP 


Weapon  and  Ammunition  Code 

RDT&E  SVCS&CONST  SUP&EQUIP  UNCODED 


Electronics  Code 

SVCS&CONST  UNCODED  RDT&E  SUP&EQUIP 


Other  Code  (includes  uncoded) 

RDT&E  UNCODED  SUP&EQUIP  SVCS&CONST 


Figure  9  Tukey  Analysis  for  SEC  Codes 
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equipment/industry  classifications  except  for  within  the 
Weapons  and  Ammunition  classification.  It  is  not 
significantly  different  than  any  other  contracting  category 
within  the  Weapons  and  Ammunition  classification.  This 
information,  combined  with  the  results  of  the  confidence 
interval  analysis,  indicates  that  the  services  and 
construction  category  has  a  significantly  higher  amount  of 
dollars  (in  terms  of  a  percentage  of  total  dollars)  going  to 
SDBs.  The  AFSC  might  perform  most  of  the  "operational"  or 
base  level  contracting  at  it's  own  bases  and  product 
divisions.  This  operational  contracting  would  likely 
include  services  and  construction  at  these  AFSC  bases  and 
product  divisions.  This  could  explain  the  higher  means  for 
this  type  of  contracting. 

The  RDT&E  and  the  Supply  and  Equipment  categories  only 
differ  significantly  within  the  "Other"  equipment/industry 
classification.  This  indicates  the  amount  of  dollars  going 
to  SDBs  (once  again  in  terms  of  percentage  of  total  dollars) 
is  not  significantly  different  in  these  classifications. 

The  AFSC  is  most  likely  to  perform  supply  and  equipment 
contracting  simultaneously  with  its  RDT&E  contracting 
because  it  is  developing  and  producing  these  products  to  fit 
its  developing  major  systems. 

Within  the  Missile  and  Rocket  industry  classification, 
the  Services  and  Construction  category  differs  significantly 
from  the  RDT&E  contracting  category  and  the  Supplies  and 
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Equipment  contracting  category  but  does  not  significantly 
differ  from  the  uncoded  entries.  This  could  be  a  result  of 
the  large  variance  associated  with  the  uncoded  data  points. 
With  a  large  variance,  it  is  more  likely  to  be  equitable 
with  the  other  treatments.  The  significant  difference 
between  the  Services  &  Construction  category  and  the  RDT&E 
and  Supplies  6  Equipment  categories  shows  that  in  the 
Missile  and  Rocket  industry  classification  most  of  the  SDB 
contracting  is  accomplished  in  the  area  of  services  and 
construction.  This  indicates  a  greater  opportunity  for  an 
SDB  to  contract  with  the  AF  in  the  Services  and  Construction 
classification  if  the  SDB  is  operating  in  the  Electronics  or 
Missile  and  Rocket  categories. 

Within  the  Weapons  and  Ammunition  classification  area, 
all  the  areas  studied  have  approximately  the  same  SDB 
efforts.  The  zero  means  and  the  small  standard  deviations 
for  all  the  contracting  categories  within  this 
classification  indicate  little  opportunity  for  SDB  firms  to 
perform  work  in  the  systems  area.  Perhaps  there  are  very 
few  or  no  SDBs  which  can  operate  in  this  area  or  perhaps  it 
is  a  result  of  the  majority  of  the  DODs  efforts  in  this  area 
being  accomplished  at  government  facilities  by  either 
government  personnel  or  employees  of  large  defense 
contractors . 

Within  the  Electronics  contracting  efforts,  none  of  the 
areas  analyzed  differ  from  the  xincoded  area,  but  the  RDT&E 
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category  and  the  Supply  &  Equipment  category  differ 
significantly  from  the  Services  &  Construction  category. 

This  is  similar  to  the  Missile  and  Rocket  categories's 
results  and  could  be  for  the  same  reasons. 

Within  the  "Other"  industry  classification,  RDT&E, 
Supplies  and  Equipment,  and  Services  and  Construction 
categories  differ  significantly  from  each  other.  Only  the 
uncoded  category  does  not  differ  significantly  from  any  of 
the  others.  Since  this  area  has  the  most  data  of  any  area, 
the  results  may  be  the  roost  accurate.  However,  they  cannot 
be  associated  with  any  specific  type  of  Supply  &  Equipment 
area  of  contracting. 

The  Tukey  analysis  with  the  FSC  code  as  the  major  sort 
factor  follows. 

FSC  code  as  Major  Factor.  The  results  of  the  Tukey 
Analysis  with  the  contracting  categories  as  the  major 
factors  are  shown  in  Figure  10. 

This  measure  of  analysis  is  useful  because  it  sorts  by 
type  of  industry.  One  can  determine  which  industry  type  (as 
determined  by  SEC  code)  has  the  greatest  percentage  of 
dollars  going  to  SDBs  in  each  contracting  category. 

Within  the  RDT&E  contracting  efforts,  the  "Other" 
category  differs  significantly  from  Missiles  &  Rockets  and 
Electronics.  However,  the  Missiles  and  Rockets, 

Electronics,  Aircraft  and  Engines,  and  Weapons  and 
Ammunition  classifications  do  not  significantly  differ.  As 
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mentioned  earlier,  the  "Other"  classification  does  not  give 
much  meaningful  information.  Therefore,  the  data  do  not 
suggest  the  four  classifications  are  different  in  their 
ability  to  contract  with  SDBs.  This  indicates  that  in  terms 
of  percentage  of  dollars  going  to  SDBs,  in  the  RDT&E 
contracting  category,  all  the  product  divisions  are 
clustered  closely  (compared  to  the  other  contracting 

RDT&E  Code 

MSL&RKT  ELEC  AIR&ENG  WPN&AMMO  OTHER 


Services  and  Construction  Code 

AIR&ENG  MSL&RKT  WPN&AMMO  ELEC  OTHER 

Supply  and  Equipment  Code 

AIR&ENG  MSL&RKT  WPN&AMMO  ELEC  OTHER 


Uncoded 

AIR&ENG  MSL&RKT  WPN&AMMO  ELEC  01. -^ER 


Figure  10  Tukey  Analysis  for  FSC  Codes 
categories)  around  the  five  percent  goal. 

Within  the  Services  and  Construction  contracting 
efforts,  the  Aircraft  and  Engine  SDB  percentages  differs 
significantly  than  the  other  areas.  Therefore,  service  and 
construction  contractors  tend  to  be  SDBs  more  in  the 
category  of  Aircraft  and  Aircraft  Engines  than  in  the  other 
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categories  studied.  Aircraft  and  aircraft  engine 
development  and  production  may  require  more  accompanying 
service  and/or  construction  efforts  than  other  types  of  AFSC 
efforts . 

Within  the  Supply  and  Equipment  contracting  category, 
the  "Other"  classification  differs  significantly  from  all 
the  other  classifications  and  Aircraft  &  Engine  differs 
significantly  from  Electronics.  The  "Other"  category  tends 
to  have  higher  percentages  of  SDB  contracting  efforts  while 
Aircraft  &  Engine  areas  tend  to  contract  with  SDBs  more  than 
Electronic  areas.  Once  again,  the  "Other"  category  does  not 
give  much  meaningful  information.  However,  there  is  a 
significant  difference  between  the  dollar  percentages  going 
to  SDBs  between  the  Aircraft  and  Aircraft  Engine  and  the 
Electronics  contracting  categories.  This  indicates  that 
within  the  Supply  and  Equipment  industry,  there  is  more  SDB 
contracting  in  the  Aircraft  and  Aircraft  Engine  category 
than  in  the  Electronics  category.  The  reason  for  this  was 
addressed  earlier  when  it  was  mentioned  that  in  the  aircraft 
and  aircraft  engine  efforts,  supplies  and  equipment  to 
support  those  systems  may  be  compatible  with  SDB 
qualifications  whereas  in  the  electronics  efforts  the 
supplies  and  equipment  may  not  be  suitable  for  SDB  efforts. 

Within  the  uncoded  area  of  contracting  efforts,  none  of 
the  areas  differ  significantly.  Since  all  the  means  of  the 
coded  classifications  were  all  equal,  the  differences 
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between  them  will  not  be  significant.  And,  since  the 
standard  deviation  for  the  ''Other"  category  was  large  with  a 
mean  near  the  others,  this  category  will  not  be 
significantly  different,  either. 

Conclusion 

The  results  of  the  statistical  analysis  show  that  there 
is  some  disparity  in  AF  organization's  abilities  to  meet  the 
five  percent  goal.  These  abilities  depend  on  both  the 
contracting  category  and  equipment/ industry  classification 
the  organization  is  operating  in.  It  also  shows  that  within 
the  contracting  categories  and  equipment  classifications, 
some  combinations  are  significantly  different  than  others  in 
their  SDB  contracting  efforts.  The  next  section  will 
address  how  these  results  can  be  generalized  to  assess  the 
ability  of  AFSC  organizations  to  meet  the  goal. 
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This  study  assessed  the  ability  of  Air  Force  Systems 
Command  (AFSC)  organizations  to  meet  a  Congressional ly 
mandated  five  percent  Small  and  Disadvantaged  Business  (SDB) 
contracting  goal .  It  achieved  this  objective  by  answering 
the  following  two  research  questions: 

1.  Does  the  percentage  of  contract  dollars  going  to 
SDBs  meet  the  five  percent  goal,  given  the  type  of 
contracting  effort  and  the  specific  industry  or  product 
classification? 

2.  Does  the  percentage  of  contract  dollars  going  to 
SDBii  significantly  vary  between  category  of  contracting 
effort  and  equipment/industry  classification? 

This  chapter  briefly  reviews  the  methodology  used  to  answer 

these  research  questions,  presents  the  conclusions  of  the 

research,  discusses  the  implications  of  the  conclusions,  and 

provides  some  recommendations  for  further  research. 


The  study  answered  the  research  questions  by  gathering 
data  and  performing  a  statistical  analysis  on  it.  Data  on 
contract  awards  to  SDB  firms  was  gathered  from  the  Air  Force 
Systems  Command  (AFSC)  contract  database  for  the  years  1987- 
1989.  It  was  organized  by  factors  based  on  the  Federal 
Supply  Classification  (FSC)  code  which  indicates  the  type  of 


contracting  effort  involved  and  the  Supply  and  Equipment 
(SEC)  code,  which  identifies  the  nature  of  the 
equipment/industry  involved.  The  contracting  categories 
studied  were  (1)  Research,  Development,  Test,  and 
Engineering  (RDT&E),  (2)  Services  and  Construction,  (3) 
Supplies  and  Equipment,  and  (4)  Uncoded.  Aircraft  and 
Aircraft  Engines,  Missiles  and  Rockets,  Weapons  and 
Ammunition,  Electronics,  and  Other  (which  included  vehicles 
and  ships)  classifications  were  chosen  to  represent  the 
industry  involved  or  product  produced.  Since  the  data  was 
gathered  from  only  the  AFSC,  the  conclusions  drawn  only 
pertain  to  AFSC  and  not  necessarily  to  other  AF,  DOD,  or 
Federal  agencies.  Generalizations  might  be  possible  to  the 
degree  that  other  DOD  organizations  undertake  the  same  type 
of  contracting  actions  with  the  same  equipment/industrial 
classifications.  However,  future  research  will  need  to 
determine  that  for  sure. 

The  statistical  analysis  consisted  of  a  confidence 
interval  analysis  and  two-factor  Analysis  of  Variance 
(ANOVA)  procedures.  The  95  percent  confidence  intervals 
were  generated  to  determine  if  the  individual  contracting 
sectors  represented  by  combinations  of  FSC  and  SEC  codes 
were  successful  in  meeting  the  five  percent  SDB  goal .  The 
ANOVA  procedure  determined  significant  interaction  between 
the  main  factor  categories  was  present  and  therefore 
pairwise  comparisons  of  means  were  conducted  between 
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treatments  within  each  factor  level.  Analysis  of  the  dat. 
determined  that  the  data  violated  imderlying  assumptions  for 
the  confidence  interval  analysis  and  the  ANOVA.  The  data 
were  not  normally  distributed,  nor  were  they  continuously 
distributed.  There  was  also  no  homogeneity  of  variance. 

Thus,  the  reliability  of  the  tests  ure  uncertain.  However, 
there  is  no  better  model  at  this  time  to  perform  the 
analysis.  Thus,  the  statistical  tests  were  used  to  gain 
whatever  insight  was  possible  into  the  research  questions. 

Research  Question  t  1 

The  confidence  interval  analysis  suggested  that  in  the 
systems  contracting  area,  an  AF  organization's  ability  to 
meet  the  five  percent  goal  which  depends  on  the  nature  of 
the  contracting  effort  and  the  type  of  equipment  or  industry 
involved.  In  general,  the  Weapon  and  Ammunition 
classification  was  the  only  factor  level  to  fall  short  of 
meeting  the  goal.  No  type  of  contracting  effort  in  this 
classification  had  contract  awards  to  SDB  firms. 

Contributing  to  this  may  be  that  many  weapon  and  ammunition 
system  development  and  production  efforts  are  performed  by 
large  businesses  using  government  facilities.  The  results 
infer  there  is  currently  little  chance  of  contracting  with 
an  SDB  in  this  area. 

The  uncoded  contracting  category  also  resulted  in  low 
levels  of  SDB  contracting  which  did  not  meet  the  goal. 
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However,  because  it  was  uncoded  inferences  based  on  this 
information  are  extremely  limited. 

The  "Other"  equipment/ industry  classification  had  many 
datapoints  (69  percent  of  the  total  datapoints)  but  the 
information  gathered  from  this  area  is  also  limited.  The 
contracting  categories  in  this  factor  level  (except  uncoded) 
all  exceeded  the  goal.  This  can  lead  to  the  conclusion 
there  are  opportunities  in  the  AFSC  to  contract  with  SDBs 
but  the  specific  classifications  of  these  opportxinities  are 
unknown.  To  the  degree  that  uncoded  entries  represent 
errors,  (i.e.,  they  should  have  been  coded  into  one  of  the 
other  classifications)  the  actual  proportions  of  contracts 
going  to  SDBs  for  the  other  SEC  areas  may  be  understated. 
Otherwise,  the  miscellaneous  categories  falling  into  the 
"Other"  area  provide  a  great  opportunity  to  contract  with 
SDBs. 

The  Services  and  Construction  category  showed  the  most 
promise  for  contracting  with  SDBs.  It  had  a  large  number  of 
datapoints  and  its  means  (except  for  in  the  Weapons  and 
Ammunition  classification)  were  higher  than  the  other 
contracting  categories.  This  implies  that  not  only  are 
there  many  opportunities  for  contracting  with  SDBs  in  the 
Services  and  Construction  area,  but  the  ability  of  SDBs  to 
receive  the  contracts  is  high.  This  category  had  many 
contracts  where  a  large  percentage  of  the  contract  went  to 
SDBs.  With  the  exception  of  the  weapons  and  ammunition 
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classification,  this  area  is  ideal  for  SDB  contracting. 

This  suggests  the  five  percent  goal  is  too  small  in  this 
area.  Construction  was  one  area  Congress  expected  to  be 
able  to  meet  a  forty  percent  goal  if  the  field  was  expanded 
to  include  small  businesses.  The  data  suggest  that  this 
might  be  an  attainable  goal  for  aircraft  related 
construction  efforts. 

In  general,  the  other  categories  and  classifications  met 
or  exceeded  the  SDB  goal.  The  one  exception  was  the  Supply 
and  Equipment  category  within  the  Electronics  industry  which 
averaged  no  SDB  actions.  There  must  be  some  unique 
characteristic  about  this  category,  at  least  for  systems 
contracting,  which  does  not  allow  it  to  cater  to  the  SDBs. 
Perhaps  there  are  too  few  SDB  producers  of  highly  technical 
electronic  systems  and  parts  typically  used  by  the  AP.  In 
contrast,  RDTSE  contracting  in  the  electronics  area 
contracted  with  SDBs  in  nearly  18  percent  of  their  efforts. 
Apparently,  there  is  great  opportunity  to  contract  with  SDBs 
in  this  area,  perhaps  due  to  a  large  percentage  of  the  PhD's 
in  this  country  being  held  by  minority  individual  as 
suggested  in  the  literature  review. 

The  data  suggests  that  when  categorized  by  (1)  Federal 
Supply  Classification  code  and  (2)  Supply  and  Equipment 
codes,  there  are  pockets  of  contracting  where  it  is 
extremely  difficult  to  meet  the  SDB  goal.  Similarly,  there 
are  pockets  of  contracting  where  the  goal  is  quite 
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attainable.  However,  the  analysis  focused  on  each  sector  to 
meet  the  five  percent  SDB  goal.  The  next  research  question 
looked  for  relationships  between  FSC  and  SEC  classifications 
as  they  relate  to  SDB  contracting. 

Research  Question  *  2 

The  pairwise  comparison  of  proportion  of  SDB  contracts 
concluded  there  were  significant  differences  in  the  ability 
of  some  contracting  categories  and  equipment/industry 
classifications  to  meet  the  five  percent  goal.  The  most 
significant  finding  was  that  the  Services  and  Construction 
contracting  category  as  a  whole  was  significantly  higher 
than  other  contracting  categories  in  all  the  industry 
classifications  except  the  Weapons  and  Ammunition 
classification.  The  significant  difference  confirms  the 
earlier  suggestion  that  this  category  has  a  significantly 
higher  percentage  of  the  contract  dollars  going  to  SDBs  than 
any  other  category  of  contracting.  The  Services  and 
Construction  could  have  a  significantly  higher  nximber  of 
SDBs  available  for  contracting. 

The  RDT&E  category  and  the  Supply  and  Equipment  category 
do  not  differ  significantly  except  when  the  "Other"  SEC 
classification  is  addressed.  This  implies  AFSC  may  have 
roughly  the  same  opportunity  of  meeting  the  SDB  goal  in 
these  categories  of  contracting  no  matter  what  industry  or 
product  they  are  dealing  with. 
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The  percentage  of  contract  dollars  going  to  SDBs  within 
the  Services  and  Construction  contracting  category  is 
significantly  higher  in  the  Aircraft  and  Aircraft  Engine 
classification.  It  may  be  that  contracting  for  the 
development  and  production  of  aircraft  and  production  of 
aircraft  systems  provides  for  more  construction  and  service 
contracting  than  does  contracting  for  other  systems.  It  is 
also  likely  there  are  more  SDB  contractors  in  the 
construction  service  area  in  general ,  and  perhaps  in  the 
aerospace  sector  in  specific. 

Within  the  Supply  and  Equipment  contracting  category, 
the  percentage  of  dollars  going  to  SDBs  is  significantly 
higher  in  the  Aircraft  and  Aircraft  Engine  industry  than  in 
the  Electronics  industry.  Perhaps  systems  contracting  for 
aircraft  and  aircraft  engines  within  the  Supply  and 
Equipment  category  uses  technology  and  skills  that  are  more 
readily  available  to  SDB  firms  than  is  the  case  for  highly 
technical  electronics  equipment.  Thus,  differences  here  may 
be  very  industry  specific. 

These  results  show  that  in  some  of  the  areas  studied, 
the  differences  between  the  treatments  are  statistically 
significant.  The  implications  of  these  will  be  discussed 
next . 

Implications  for  Managers 

One  may  notice  the  equipment/industry  classifications 
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correspond  to  the  product  divisions  of  AFSC.  The  Aircraft 
and  Aircraft  Engine  contracting  is  performed  primarily  by 
the  Aeronautical  Systems  Division  (ASD),  the  Missile  and 
Rocket  contracting  by  the  Space  Systems  Division  (SSD)  and 
the  Ballistic  Systems  Missile  Division  (BSD),  the  Weapons 
and  Ammunition  contracting  by  the  Munitions  Systems  Division 
(MSD)  and  the  Electronics  contracting  by  the  Electronic 
Systems  Division  (ESD).  Therefore  conclusions  drawn  about 
the  equipment/industry  classifications  may  be  appropriate 
for  the  respective  AFSC  divisions. 

It  appears  as  if  the  MSD  had  very  few  contracting 
opportunities  with  SDBs.  However,  the  MSD  contracts  deal 
primarily  with  highly  sophisticated  munitions  which  take 
years  to  develop  and  produce.  Many  of  these  are  produced  by 
large  contractors  at  government  facilities  and  plants. 

These  complicated  systems  leave  little  opportunity  for  SDBs 
to  perform  the  prime  contracts,  but  may  leave  considerable 
sub-contracting  opportunities.  It  also  might  be  that  the 
greatest  opportunity  for  SDB  contracting  in  this  area  would 
be  replenishment  spares.  In  order  to  better  analyze  the 
Weapons  and  Ammunition  classification,  one  should  also  study 
Air  Force  Logistics  Command  (AFLC),  Army,  and  Navy  contracts 
which  probably  have  a  much  larger  number  of  contracts  in 
this  area.  In  order  to  improve  SDB  participation,  MSD  would 
have  to  develop  capable  SDB  sources  that  could  help  develop, 
produce,  or  test  the  type  of  weapon  systems  and  AF  munitions 
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it  manages. 

The  ESD  had  better  opportunities  for  SDB  contracting  in 
the  RDT&E  area  than  in  the  supplies  and  equipment  area. 

This  imlies  that  ESD  should  use  a  search  strategy  to  find 
SDBs  who  can  perform  RDT&E  efforts.  However,  However,  in 
the  supplies  and  equipment  area,  ESD  should  focus  on 
developing  sources,  if  possible,  to  increase  the 
opportunities  in  this  area. 

The  ASD  had  high  opportunity  for  SDB  contracting  in  all 
its  areas.  The  reasons  for  this  could  be  analyzed  and  the 
conclusions  could  be  used  to  develop  a  strategy  for 
increasing  SDB  contracting  support  other  divisions  to  meet 
the  SDB  goal.  The  SSD  and  BSD  could  likewise  develop  a 
strategy  for  increasing  SDB  contracting  in  their  areas. 

The  confidence  interval  analysis  indicated  that  all  the 
rest  of  the  divisions  in  the  AFSC  met  or  exceeded  the  goal 
in  the  various  categories  except  for  within  the  Uncoded 
category  and  the  Supply  and  Equipment  category  in  the 
classification  related  tu  the  ESD.  It  appears  there  is 
sufficient  opportunity  to  contract  with  SDBs  in  the  RDT&E 
and  the  Services  and  Construction  areas  at  ESD,  but  the 
Supply  and  Equipment  category  indicates  there  is  no 
opportunity  to  contract  with  SDBs.  The  Supply  and  Equipment 
contracting  at  the  ESD  must  have  unique  characteristics 
which  keeps  it  from  contracting  with  SDBs.  Perhaps  the 
systems  developed  and  produced  at  the  ESD  do  not  require 
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parts  compatible  with  SDB  abilities. 

The  greatest  area  for  SDB  contract  potential  in  AFSC  is 
in  the  Services  and  Construction  category  and  especially  at 
the  ASD.  There  are  definitely  some  unique  characteristics 
of  both  these  factor  levels  which  allow  SDB  contracts  to 
flourish.  The  worst  area  for  SDB  contract  potential  in  the 
AFSC  is  in  the  Supply  and  Equipment  category  at  the  ESD. 

SDB  potential  in  the  area  of  Weapons  and  Ammunition  also 
appears  weak  in  the  AFSC. 

Based  on  this  study,  an  organization  which  deals 
primarily  in  an  area  such  as  weapons  and  ammunition  should 
not  be  mandated  to  meet  the  same  goal  for  contracting  with 
SDBs  as  other  areas  such  as  RDTSE  because  there  are 
significant  differences  in  the  ability  to  meet  the  goal, 
based  on  the  type  of  contracting  effort  and  type  of  industry 
involved.  Perhaps  the  solution  is  to  make  the  goal 
particular  to  specific  contracting  groups  based  on  the 
amount  of  SDB  contracting  opportunity.  This  is  one  area 
which  warrants  further  research.  The  next  section  discusses 
more  possible  areas  for  further  research. 

Recommendations  for  Future  Research 

This  study  only  looked  at  AFSC  data  for  the  years  1987- 
1989.  Further  research  should  include  at  least  Air  Force 
Logistics  Command  (AFLC)  data  since  they  perform  most  of  the 
central  procurement  type  contracting  (replenishment  spares. 
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modifications,  sustaining  engineering,  repairs,  overhauls, 
etc.)-  Ultimately,  data  from  the  entire  AF,  if  not  the  DOD, 
should  be  analyzed  in  a  similar  fashion  to  obtain  more 
accurate  and  generalizable  results.  More  years  should  be 
analyzed  (including  those  prior  to  1987)  in  order  to  gather 
trend  data  to  see  if  the  percentages  of  SDB  contracts  have 
greatly  increased.  This  would  determine  whether  the  mandate 
has  had  any  effect  on  the  percentages. 

More  detailed  analysis  should  be  conducted  in  the  areas 
where  high  and  low  percentages  of  SDB  contracting  exist. 

This  will  provide  insight  into  the  \inique  characteristics  of 
these  areas  to  identify  the  factors  that  encourage  or 
restrain  SDB  participation.  This  will  identify  ways  to 
increase  SDB  participation  in  areas  previously  restricted. 
Further  research  into  the  reasons  why  the  differences  exist 
is  warranted. 

The  conclusions  had  many  speculations  as  to  why  the 
differences  existed.  If  the  reasons  for  the  differences  can 
be  determined  through  analysis,  the  differences  may  be  able 
to  be  minimized  so  SDBs  from  all  areas  of  Federal 
contracting  can  be  used  to  the  greatest  possible  extent. 
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CASES  INCLUDED  507  MISSING  CASES 
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CASES  INCLUDED  46  MISSING  CASES 
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CASES  INCLUDED  53  MISSING  CASES 
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CASES  INCLUDED  763  MISSING  CASES 
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CASES  INCLUDED  186  MISSING  CASES 
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CASES  INCLUDED  4  MISSING  CASES 
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CASES  INCLUDED  66  MISSING  CASES 
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CASES  INCLUDED  674  MISSING  CASES 


HISTOGRAM  OF  CELL16 
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CASES  INCLUDED  1  MISSING  CASES 


HISTOGRAM  OF  CF.LL17 


X 

6> 

® 

® 

® 

® 

® 

® 

® 

® 

® 

C£) 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

♦-H 

1  tH 

00 

uo 

vO 

CO 

c^ 

® 

fH 

I 

s> 

® 

® 

® 

® 

® 

® 

® 

® 

CM  ® 

® 

® 

® 

®  ® 

® 

® 

® 

Z 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

3 

G> 

® 

® 

® 

® 

® 

® 

® 

® 

® 

® 

O 

(Ni 

00 

■c 

in 

o 

tv 

00 

O' 

® 

G> 

® 

® 

® 

® 

® 

® 

® 

® 

® 

103 


CASES  INCLUDED  2  MISSING  CASES 
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CASES  INCLUDED  1  MISSING  CASES 


HISTOGRAM  OF  CELLIR 
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CASES  INCLUDED  2  MISSING  CASES 
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CASES  INCLUDED  6  MISSING  CASES 


RANKITS  VS  CELLl 


RANKITS 
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RANKITS  VS  CELLS 
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APPgn<aiy  B: _ QPATTRO  Output 


Confidence  Intervals  Generated  by  QUATTRO 
Across  FSC  codes  and  SEC  A&E  code 


-0.3104 

<  = 

mu . 1  -  mu . 2 

<  = 

-0.2074 

-0.0381 

<  = 

mu.l  -  mu. 3 

<  = 

0.0321 

-0.1050 

<  = 

mu . 1  -  mu . 4 

<  = 

0.2395 

7.8813 

<  = 

mu . 2  -  mu . 3 

<  = 

7.9783 

0.0284 

<  = 

mu . 2  -  mu . 4 

<  = 

0.6237 

-0.2250 

<  = 

mu . 3  -  mu . 4 

<  = 

0.3655 

Across  FSC  codes  and  SEC  M&R  code 


-0.1545 

<  = 

mu.l  -  mu . 2 

<  = 

-0.0629 

-0.0385 

<  = 

mu.l  -  mu . 3 

<  = 

0.0346 

-0.1051 

<  = 

mu.l  -  mu . 4 

<  = 

0.1896 

0.0573 

<  = 

mu . 2  -  mu . 3 

<  = 

0.1562 

-0.0589 

<  = 

mu . 2  -  mu . 4 

<  = 

0.3608 

-0.1638 

<  = 

mu . 3  -  mu . 4 

<  = 

0.2522 

-0.3042 

<S 

mu.l  -  mu . 2 

<  = 

0.3042 

Across  FSC  codes  and  SEC  W&A  code 


-0.2484 

<  = 

mu.l  -  mu . 3 

<  = 

0.2484 

-0.2683 

<  = 

mu.l  -  mu . 4 

<  = 

0.2683 

-0.3042 

<  = 

mu . 2  -  mu . 3 

<  = 

0.3042 

-0.4302 

<  = 

mu . 2  -  mu . 4 

<  = 

0.4302 

-0.3927 

<  = 

mu . 3  -  mu . 4 

<  = 

0.3927 

Across  FSC  code  and  SEC  E  code 


-0.2023 

<  = 

mu.l  -  mu . 2 

<  = 

-0.0976 

-0.0086 

<  = 

mu.l  -  mu . 3 

<  = 

0.0854 

-0.1342 

<  = 

mu.l  -  mu . 4 

<  = 

0.2111 

0.1322 

<  = 

mu . 2  -  mu . 3 

<  = 

0.2446 

-0.1092 

<  = 

mu . 2  -  mu . 4 

<  = 

0.4860 

-0.2967 

<  = 

mu . 3  -  mu . 4 

<  = 

0.2967 

Across  FSC  code  and  SEC  O  code 


-0.0942 

<  = 

mu.l  -  mu . 2 

<  = 

-0.0608 

-0.0604 

<  = 

mu.l  -  mu . 3 

<  = 

-0.0262 

-0.0347 

<  = 

mu.l  -  mu . 4 

<  = 

0.0494 

0.0188 

<  = 

mu . 2  -  mu . 3 

<  = 

0.0496 

-0.0345 

<  = 

mu . 2  -  mu . 4 

<  = 

0.2043 

-0.0688 

<  = 

mu . 3  -  mu . 4 

<  = 

0.1701 

Across  SEC  codes  and  FSC  RDT&E  code 


-0.0103 

<  = 

mu.  1 

- 

mu.  2 

<  = 

0.0603 

-0.0826 

<  = 

mu.  1 

- 

mu.  3 

<  = 

0.2170 

• 

-0.0115 

<  = 

mu.  1 

- 

mu.  4 

<  = 

0.0691 

-0.0605 

<  = 

mu.  1 

- 

mu.  5 

<  = 

0.0001 

-0.1057 

<  = 

mu.  2 

- 

mu.  3 

<  = 

0.1901 

» 

-0.0332 

<  = 

mu.  2 

- 

mu.  4 

<  = 

0.0408 

-0.0809 

<  = 

mu.  2 

- 

mu.  5 

<  = 

-0.0294 

-0.1892 

<  = 

mu.  3 

- 

mu.  4 

<  = 

0.1123 

-0.2436 

<  = 

mu.  3 

- 

mu.  5 

<  = 

0.0488 

-0.0913 

<  = 

mu.  4 

- 

mu.  5 

<  = 

-0.0267 

Across  SEC  codes  and  FSC  S&C  code 


0.1156 

<  = 

mu.  1 

- 

mu.  2 

<  = 

0.2347 

0.1088 

<  = 

mu.  1 

- 

mu.  3 

<  = 

0.5433 

0.0764 

<  = 

mu.  1 

- 

mu.  4 

<  = 

0.1990 

0.1069 

<  = 

mu.  1 

- 

mu.  5 

<  = 

0.1954 

-0.0634 

<  = 

mu.  2 

- 

mu.  3 

<  = 

0.3653 

-0.0967 

<  = 

mu.  2 

- 

mu.  4 

<  = 

0.0217 

-0.0653 

<s 

mu.  2 

- 

mu.  5 

<  = 

0.0174 

-0.4056 

<  = 

mu.  3 

- 

mu.  4 

<  = 

0.0288 

-0.3812 

<S 

mu.  3 

- 

mu.  5 

<  = 

0.0313 

-0.0303 

<  = 

mu.  4 

- 

mu.  5 

<  = 

0.0573 

Across  SEC  codes  and  FSC  S&E  code 
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<  = 

mu.  1 
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mu.  2 

<  = 
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<  = 

mu.  1 
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mu.  3 

<  = 
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<  = 

mu.l 

- 

mu.  4 

<  = 
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<  = 

mu.l 

- 

mu.  5 

<  = 
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-0.1043 

<  = 

mu.  2 
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mu.  3 

<  = 
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-0.0019 

<  = 

mu.  2 

- 

mu.  4 

<  = 
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<  = 

mu.  2 

- 

mu.  5 

<  = 

-0.0654 
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<  = 

mu.  3 

- 

mu.  4 

<  = 

0.1533 

-0.2868 

mu.  3 

- 

mu.  5 

<  = 

0.0054 

-0.1783 

<  = 

mu.  4 

- 

mu.  5 

<  = 

-0.1031 

Across  SEC  codes  and  FSC  uncoded  code 


-0.5796 

<  = 

mu.l 

- 

mu.  2 

<  = 

0.5796 

-0.9043 

<  = 

mu.l 

- 

mu.  3 

<  = 

0.9043 

-0.9043 

<  = 

mu.l 

- 

mu.  4 

<  = 

0.9043 

-0.4695 

<  = 

mu.l 

- 

mu.  5 

<  = 

0.2894 

-0.5796 

<  = 

mu.  2 

- 

mu.  3 

<  = 

0.5796 

-0.5796 

<  = 

mu.  2 

- 

mu.  4 

<  = 

0.5796 

-0.3700 

<  = 

mu.  2 

- 

mu.  5 

<  = 

0.1899 

-0.9043 

<  = 

mu.  3 

- 

mu.  4 

<  = 

0.9043 

-0.4695 

<  = 

mu.  3 

- 

mu.  5 

<  = 

0.2894 

-0.4695 

<  = 

mu.  4 

- 

mu.  5 

<  = 

0.2894 
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